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Abstract 

This  thesis  presents  the  results  of  a  handling  qualities  evaluation  of  a  supersonic 
tailless  air  vehicle.  The  2006  Quadrennial  Defense  Review  mandated  the  need  for  the 
next  generation  of  long-range  strike  aircraft  by  2018.  Due  to  speed  and  stealth 
requirements,  this  resulted  in  a  tailless  aircraft  with  an  instantaneous  center  of  rotation 
located  well  forward  of  that  of  a  conventional  aircraft.  This  thesis  examines  how  this 
center  of  rotation  affected  pilot  handling  qualities  ratings.  This  effect  should  have  been 
the  most  pronounced  during  approach  and  landing,  and  was  where  the  testing  focused. 
The  goal  of  this  research  was  to  develop  a  systematic  procedure  for  evaluating  the 
handling  qualities  of  this  aircraft,  and  to  determine  how  different  pilot  flying  techniques 
or  pilot-inceptor  interactions  influenced  them.  This  procedure  was  demonstrated  in 
simulator  testing  and  in  flight  testing  on  the  Calspan-operated  Total  In-Flight  Simulator 
aircraft. 
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HANDLING  QUALITIES  EVALUATION  OF  A  SUPERSONIC  TAILLESS  AIR 

VEHICLE 
1.0  Introduction 

1.1  Purpose 

Since  the  dawn  of  heavier-than-air  flight  just  over  a  century  ago,  man  has 
attempted  to  qualify  and  quantify  his  experience  in  the  air.  What  began  as  discussion 
between  the  two  Wright  brothers  on  how  to  improve  their  flying  machine  developed  into 
the  methods  used  by  modern  test  pilots  to  describe  a  new  aircraft.  As  the  United  States 
Air  Force  modernizes  during  the  first  part  of  the  21st  century,  it  will  continue  to  test  and 
evaluate  new  concept  aircraft  to  detennine  which  will  best  satisfy  mission  requirements. 
The  handling  qualities  evaluation  is  part  of  this  test  and  evaluation  process.  In  simple 
tenns,  handling  qualities  describe  the  characteristics  or  dynamics  of  both  the  pilot  and 
aircraft  working  together.  The  better  the  handling  qualities  (HQ)  of  an  aircraft,  the  more 
likely  a  pilot  will  be  able  to  accomplish  the  design  mission.  The  purpose  of  this  thesis 
was  to  conduct  a  handling  qualities  evaluation  of  a  new  concept  aircraft  proposed  for  the 
next  generation  bomber:  a  supersonic  tailless  air  vehicle  (STAV).  In  addition,  it  sought 
to  determine  if  different  pilot  flying  characteristics  or  pilot-inceptor  interactions  impacted 
the  pilot’s  opinion  of  the  aircraft  handling  qualities. 

1.2  Motivation 

Every  four  years,  the  US  Department  of  Defense  (DoD)  conducts  a  Quadrennial 
Defense  Review  (QDR)  of  its  vision  and  mission.  The  2006  QDR  outlined  plans  for  a 
new  USAF  (United  States  Air  Force)  long-range  strike  aircraft  to  be  fielded  by  2018  that 
could  meet  certain  stealth  and  speed  requirements.  Several  major  defense  contractors 
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initiated  programs  designed  to  fulfill  this  new  long-range  strike  requirement.  The 
Northrop  Grumman  Corporation  (NGC)  design  was  unconventional,  and  consisted  of  a 
tailless  aircraft  that  had  a  cockpit  located  well  aft  of  a  conventional  cockpit  location.  The 
unique  aspects  of  this  Supersonic  Tailless  Air  Vehicle  (STAV)  resulted  in  an 
instantaneous  center  of  rotation  (ICR)  that  was  nearly  collocated  with  the  cockpit.  This 
meant  that  the  initial  flight  path  response  to  a  given  pitch  input  would  be  opposite  the 
direction  of  the  input,  an  effect  most  pronounced  to  the  pilot  during  approach  and 
landing.  This  thesis  research  focuses  on  the  unique  handling  qualities  characteristics  of 
the  STAV  during  approach  and  landing. 

1.3  Research  Objectives 

The  primary  objective  for  this  thesis  was  to  evaluate  the  handling  qualities  of  the 
NGC  STAV  model  and  its  flight  control  system  during  the  powered  approach  and 
landing  phase  of  flight,  an  objective  supported  by  the  various  individual  research 
objectives  of  three  distinct  test  sections.  These  sections  included  research  in  the  Infinity 
Cube  Simulator  (ICS),  Large  Amplitude  Multi-mode  Aerospace  Research  Simulator 
(LAMARS),  and  the  variable-stability  Total  In-Flight  Simulator  (TIFS)  aircraft.  The  first 
two  objectives  involved  the  ICS  testing,  objectives  three  through  five  applied  to 
LAMARS  testing.  The  sixth  objective  was  used  in  both  LAMARS  and  TIFS  testing,  and 
the  final  two  objectives  concerned  only  the  TIFS  flight  testing. 

1.3.1  Objective  1  -  Determine  if  piloting  technique  or  background  influenced 
how  an  aircraft’s  handling  qualities  were  rated.  This  objective  sought  to  reveal  any 
differences  in  handling  qualities  ratings  that  resulted  from  different  piloting  backgrounds. 
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This  included  differences  between  pilots  in  service,  type  of  aircraft  flown,  and  test 
experience. 

1.3.2  Objective  2  -  Establish  an  overall  test  methodology  to  use  in  both 
simulator  and  flight  testing  at  USAF  Test  Pilot  School  (TPS).  The  overall  test 
methodology  had  to  be  conducive  to  both  research  at  the  Air  Force  Institute  of 
Technology  (AFIT)  and  to  simulator  and  flight  testing  during  TPS. 

1.3.3  Objective  3  -  Determine  the  best  feedback  control  system  (angle  of 
attack,  flight-path  angle,  or  pitch  rate)  of  the  baseline  STAV  model.  The  baseline 
STAV  model  was  the  second  version  of  the  flight  control  system  developed  by  the  NGC 
to  operate  its  supersonic  tailless  air  vehicle.  The  model  was  capable  of  feeding  back  any 
one  of  the  three  parameters,  and  the  pilots  had  to  determine  which  produced  the  best 
handling  qualities. 

1.3.4  Objective  4  -  Determine  the  baseline  STAV  model  flying  qualities  as 
implemented  on  LAMARS.  In  order  to  make  sure  that  the  system  under  test  in 
LAMARS  was  the  same  as  the  baseline  STAV  model,  the  flying  qualities  of  the 
LAMARS  simulation  were  compared  with  those  of  the  baseline  model.  A  good 
correlation  between  the  two  ensured  the  fidelity  of  the  simulation. 

1.3.5  Objective  5  -  Develop  an  optimized  flight  control  system,  feel  system,  or 
technique  to  flight  test  in  the  TIFS  in  addition  to  the  baseline  STAV  model.  Based 
on  the  initial  handling  qualities  results  of  previous  NGC  and  ICS  testing,  new  methods 
were  employed  to  improve  the  perceived  handling  qualities. 

1.3.6  Objective  6  -  Compare  the  LAMARS  optimized  control  system  to  the 
baseline  STAV  control  system.  The  optimized  system  that  employed  the  new  methods 
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could  then  be  compared  to  the  baseline  STAV  control  system  to  show  any  differences  in 
perceived  handling  qualities.  These  systems  were  compared  both  in  LAMARS  and  on 
TIFS. 

1.3.7  Objective  7  -  Determine  the  powered  approach  handling  qualities  of  the 
baseline  STAV  model.  The  handling  qualities  of  the  baseline  STAV  model  in  flight 
were  evaluated  on  TIFS  during  approach  and  landing. 

1.3.8  Objective  8  -  Determine  the  flying  qualities  for  the  TIFS  simulation  of 
the  STAV  flight  control  system.  In  order  to  make  sure  that  the  system  under  test  in 
TIFS  was  the  same  as  the  baseline  STAV  model,  the  flying  qualities  of  the  TIFS 
simulation  were  again  compared  with  those  of  the  baseline  model.  A  good  correlation 
between  the  two  ensured  the  fidelity  of  the  simulation. 

1.4  Research  Overview 

As  shown  in  the  previous  research  objectives,  the  research  of  this  thesis  was 
divided  into  three  distinct  test  sections.  The  first  section  included  research  conducted  at 
AFIT  prior  to  attending  TPS.  While  at  AFIT,  a  group  of  nineteen  different  Air  Force, 
Navy,  Marine  Corps,  and  civilian  pilots  conducted  simulator  testing  in  the  ICS  in  an 
effort  to  address  the  first  two  research  objectives.  Each  pilot  conducted  ten  different 
approaches  and  landings  in  different  aircraft  models  and  assigned  handling  qualities 
ratings  for  each.  The  results  were  analyzed  to  detennine  any  performance  or  ratings 
differences  between  the  various  pilots.  The  general  test  procedures  employed  in  ICS 
testing  were  used  as  a  framework  for  future  testing  during  TPS. 

During  TPS,  a  group  of  three  test  pilots  and  three  flight  test  engineers  formed  the 
HAVE  STAV  test  team  and  addressed  the  remaining  six  research  objectives  of  test 
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sections  two  and  three.  As  mentioned  in  objective  five,  full-motion  simulator  testing  in 
LAMARS  was  conducted  to  develop  an  optimized  control  system,  feel  system,  or 
technique  to  flight  test  in  TIFS  in  addition  to  the  baseline  STAV  model.  It  also  served  to 
familiarize  the  test  team  with  the  baseline  STAV  model  handling  qualities  prior  to  flight 
testing.  Additionally,  HAVE  STAV  conducted  160  different  approaches  and  landings 
over  sixteen  hours  of  simulator  testing.  The  team  then  analyzed  the  results  for  a  month 
before  the  flight  tests  were  conducted  on  TIFS  at  a  Calspan  facility  in  Niagara  Falls,  New 
York.  During  a  week  of  flight  testing  six  sorties  encompassing  sixty-seven  different 
approaches  for  data  were  flown.  The  flight  test  data  were  analyzed  and  reported  on  prior 
to  completing  TPS. 

1.5  Preview  of  Results 

There  were  several  significant  results  found  during  the  conduct  of  this  thesis.  A 
brief  synopsis  of  these  major  results  follows,  a  more  detailed  discussion  of  these  results 
can  be  found  later  on  in  this  thesis. 

The  ICS  testing  showed  that  pilot  background  had  an  impact  not  only  on  the 
handling  qualities  rating,  but  also  on  the  learning  rate  and  the  precision  used  to  complete 
flying  tasks.  Overall,  the  pilot  accuracy  correlated  well  with  the  pilot  rating,  where  the 
pilots  who  performed  the  best  generally  gave  the  best  handling  qualities  ratings.  The 
powered  approach  and  landing  tasks  developed  in  the  ICS  and  used  throughout  testing 
were  demanding  enough  to  test  both  the  flight  controls  and  the  pilots  while  remaining 
operationally  representative. 

The  LAMARS  testing  showed  that  the  angle  of  attack  (alpha-command)  control 
system  was  favored  by  the  pilots.  The  head  STAV  flight  control  engineer  confirmed  that 
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the  flying  qualities  of  the  STAV  as  implemented  on  LAMARS  were  the  same  as  those 
exhibited  during  previous  NGC  testing.  The  handling  qualities  of  the  baseline  STAV 
model  during  the  approach  (above  300  feet  AGL)  were  not  problematic,  and  were 
considered  satisfactory  by  the  pilots.  However,  once  below  this  altitude  the  handling 
qualities  degraded,  particularly  when  attempting  to  flare  the  STAV.  All  pilots  noted  that 
the  flare  was  the  most  difficult  part  of  a  landing  task.  While  the  control  system  optimized 
in  LAMARS  still  had  a  good  number  of  inadequate  landings  and  therefore  unacceptable 
handling  qualities,  it  displayed  a  marked  improvement  over  the  baseline  STAV  model. 

For  the  TIFS  flight  tests,  the  handling  qualities  of  the  baseline  STAV  model 
during  the  approach  (above  300  feet  AGL)  were  again  considered  satisfactory  by  the 
pilots.  Below  this  altitude  the  baseline  STAV  handling  qualities  remained  predominantly 
unacceptable.  The  primary  reason  for  these  poor  handling  qualities  was  not  a  high  pilot 
workload,  but  the  inability  of  the  pilots  to  meet  the  defined  performance  criteria.  Both 
the  pilot  workload  and  compensation  were  deemed  acceptable.  The  flare  was  again  noted 
as  the  most  difficult  part  of  a  landing  task.  The  most  objectionable  flight  control 
characteristic  during  a  landing  with  the  baseline  STAV  model  was  pitch  sensitivity.  The 
comparison  of  the  LAMARS  optimized  control  system  with  the  baseline  STAV  control 
system  showed  that  the  optimized  system  had  improved  handling  qualities  over  the 
baseline  system.  The  number  of  landings  which  achieved  desired  performance  nearly 
tripled,  while  the  number  of  inadequate  landings  decreased  by  thirty  percent.  Despite  this 
increase  in  performance  over  the  baseline  system,  the  optimized  system  still  had  almost 
twice  as  many  unacceptable  landings  as  satisfactory  landings.  These  results  indicated 
that  the  optimized  system,  while  better  than  the  baseline  system,  still  had  major 
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deficiencies  requiring  improvement.  Overall,  the  TIFS  aircraft  was  able  to  effectively 
match  the  flight  characteristics  represented  by  the  STAV  equations  of  motion.  This 
illustrated  the  fact  that  although  the  aircraft  did  not  yet  physically  exist,  the  STAV 
handling  qualities  could  be  detennined  using  the  TIFS. 

1.6  Thesis  Overview 

The  first  chapter  of  this  study  introduced  the  purpose  and  motivation  behind  this 
thesis.  It  outlined  the  research  objectives  of  each  test  section  and  provided  a  brief 
overview  of  the  research  conducted  during  the  thesis.  It  then  previewed  the  results  of  the 
thesis  research  before  providing  an  overview  of  the  thesis  itself. 

Chapter  2  of  this  research  contains  descriptions  of  and  background  infonnation 
about  the  assorted  topics  related  to  this  thesis  research.  It  depicts  and  explains  various 
handling  qualities  ratings  scales  including  the  Cooper-Harper  rating  (CHR)  scale,  and 
discusses  when  and  where  each  could  be  used.  It  then  details  the  impetus  for  and 
research  behind  the  next  generation  of  long-range  strike  aircraft,  focusing  on  the  2006 
QDR  and  NGC’s  STAV  design.  The  related  research  of  the  cranked-arrow  delta  wing  F- 
16XL  program  and  the  Space  Shuttle/  TIFS  program  are  then  discussed.  Several 
aerodynamic  concepts  that  impact  the  handling  qualities  during  powered  approach  are 
then  detailed,  including:  static  and  dynamic  longitudinal  and  lateral/  directional  aircraft 
stability,  dynamic  inversion  in  flight  controls,  instantaneous  center  of  rotation,  and  the 
power  required  curve.  The  two  different  pilot-in- the-loop  simulators  used  during  this 
research  are  then  described,  followed  by  the  histories  behind  several  different  variable 
stability  aircraft. 
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The  different  methods  and  procedures  for  conducting  this  study’s  test  research  are 
outlined  in  Chapter  3.  The  discussion  first  focuses  on  the  scope  and  assumptions  of  this 
thesis.  It  then  covers  the  overall  general  test  methodology,  including  the  initial  test 
procedures  developed  using  the  previous  LAMARS  testing  by  the  NGC.  It  also  describes 
the  specific  integration  of  the  different  STAV  models  and  the  test  specific  procedures  for 
each  portion  of  the  testing:  ICS  testing,  LAMARS  testing,  and  TIFS  flight  testing.  These 
test  specific  procedures  included  both  the  tasks  and  test  cards  that  each  pilot  flew  as  well 
as  the  desired  parameters  and  constraints  used  in  each  test  section.  Finally,  the  data 
analysis  plans  for  the  test  results  of  each  section  are  all  explained  in  detail. 

The  results  and  analysis  of  all  testing  are  contained  in  Chapter  4.  It  is  again 
divided  into  the  three  main  test  sections:  ICS  testing,  LAMARS  testing,  and  TIFS  flight 
testing.  It  summarizes  the  pool  of  pilots  for  the  ICS  testing  and  shows  who  participated 
in  each  portion  of  the  testing.  It  breaks  down  the  results  of  the  ICS  section  first  by 
aircraft,  then  into  overall  HQ  rating  and  data  precision,  and  finally  by  HQ  rating  and  data 
precision  according  to  pilot  classification.  For  the  LAMARS  and  TIFS  testing,  it  looks  at 
results  of  the  baseline  and  LAMARS  optimized  models,  as  well  as  the  comparison 
between  the  two.  The  results  include  pilot  performance  and  CHR  for  all  three  test 
sections  and  pilot  workload  vs.  aggressiveness  for  the  LAMARS  and  TIFS  testing.  Each 
section  discusses:  if  the  pilot  ratings  differed  according  to  classification;  ways  to  improve 
the  test  results;  and  any  issues  that  hindered  the  tests  or  proved  to  be  poor  assumptions. 

Chapter  5  is  a  summary  of  the  entire  thesis  research,  and  includes  both 
conclusions  from  the  data  and  recommendations  for  the  future.  The  chapter  is  again 
divided  into  the  three  main  test  sections:  ICS  testing,  LAMARS  testing,  and  TIFS  testing. 
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The  conclusions  are  drawn  from  the  complete  data  analysis,  and  provide  the  most  salient 
points  to  take  away  from  each  section.  The  recommendations  consist  of  things  that  can 
be  done  to  refine  or  expand  the  testing,  as  well  as  possible  areas  for  future  research  to 
explore.  The  chapter  shows  when  the  recommendations  of  one  test  section  were  used  in 
another,  as  well  as  when  they  were  not  followed  due  to  outside  constraints  or  limitations. 
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2.0  Background 


2.1  Overview 

This  chapter  contains  background  information  about  different  topics  related  to  this 
thesis  research.  It  depicts  and  explains  various  handling  qualities  ratings  scales  including 
the  Cooper-Harper  rating  (CHR)  scale,  and  discusses  when  and  where  each  could  be 
used.  It  then  details  the  impetus  for  and  research  behind  the  next  generation  of  long- 
range  strike  aircraft,  focusing  on  the  2006  QDR  and  NGC’s  STAV  design.  The  related 
research  of  the  cranked-arrow  delta  wing  F-16XL  program  and  the  Space  Shuttle/  TIFS 
program  are  then  discussed.  Several  aerodynamic  concepts  that  impact  the  handling 
qualities  during  powered  approach  are  then  detailed,  including:  static  and  dynamic 
longitudinal  and  lateral/  directional  aircraft  stability,  dynamic  inversion  flight  controls, 
instantaneous  center  of  rotation,  and  the  front  and  back  side  of  the  power  required  curve. 
The  two  different  pilot-in-the-loop  simulators  used  during  this  research  are  then 
described,  followed  by  the  histories  and  descriptions  of  several  different  variable  stability 
aircraft. 

2.2  Handling  Qualities  Rating  Scales 

Before  a  discussion  of  handling  qualities  evaluations  or  pilot  rating  scales  can 
begin,  both  concepts  need  to  be  defined  in  further  detail.  What  are  handling  qualities? 
The  terms  flying  qualities  (FQ)  and  handling  qualities  (HQ)  were  sometimes  used 
interchangeably.  In  the  1930’s  the  U.S.  Army  Air  Corps  designer’s  handbook  summed 
up  flying  qualities  specifications  in  a  single  sentence:  “The  stability  and  control 
characteristics  should  be  satisfactory”  (Liebst-MECH  629,  2006).  A  USAF  Test  Pilot 
School  (TPS)  textbook  defines  flying  qualities  as:  “Those  stability  and  control 
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characteristics  which  influence  the  ease  of  safely  flying  an  aircraft  during  steady  and 
maneuvering  flight  in  the  execution  of  the  total  mission”  (DoD-TPS,  2002).  Cooper  and 
Harper  state  that:  “Handling  qualities  are  those  qualities  or  characteristics  that  govern  the 
ease  and  precision  with  which  a  pilot  is  able  to  perform  the  tasks  required  in  support  of 
the  aircraft  role”  (Liebst-MECH  529,  2006). 

Although  the  terms  are  still  sometimes  used  interchangeably,  flying  qualities  and 
handling  qualities  are  different.  The  pilot-aircraft  system  can  be  divided  into  two 
categories:  those  times  when  the  pilot  is  out  of  the  loop  (an  open-loop  system),  and  those 
times  when  the  pilot  is  in  the  loop  (a  closed-loop  system).  For  a  completely  open-loop 
system,  where  the  pilot  is  not  included,  the  stability  and  control  characteristics  of  an 
aircraft  define  the  flying  qualities  of  that  aircraft.  The  characteristics  of  both  the  pilot  and 
aircraft  working  together  in  a  closed-loop  system  define  the  handling  qualities  of  that 
aircraft.  Flying  qualities  requirements  are  met  by  properly  modeling  and  designing  the 
flight  control  system  to  make  the  bare  airframe  appear  to  have  the  same  characteristics  as 
a  historically  desirable  aircraft.  Handling  qualities  mainly  deal  with  the  aircraft  mission 
performance,  and  have  the  most  impact  on  how  a  pilot  will  rate  an  aircraft.  Handling 
qualities  are  rated  at  three  primary  levels  (MIL-STD  1797B,  2006).  Level  1  HQ  are 
satisfactory,  and  are  adequate  to  complete  the  mission.  Level  2  HQ  are  acceptable,  but 
with  some  increasing  pilot  workload  and/  or  degradation  in  mission  performance.  Level 
3  HQ  mean  that  while  the  aircraft  is  controllable,  the  pilot  workload  is  excessive  or 
mission  effectiveness  is  inadequate. 

The  ability  to  measure  and  record  a  pilot’s  opinion  of  how  well  an  aircraft  flies  is 
vitally  important.  It  allows  a  pilot  to  evaluate  a  specific  aircraft  for  its  operational 
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suitability,  or  to  record  how  well  a  certain  configuration  performs  so  that  it  can  be  used  to 
help  future  designers  (Hodgkinson,  1998).  The  simplest  way  to  collect  pilot  opinion  is 
via  a  pilot  comment  card,  where  the  pilot  answers  questions  about  certain  flying  tasks. 
This  is  useful  when  conducting  small-scale  tests,  but  when  working  with  large  numbers 
of  test  configurations  or  pilots,  a  numerical  record  of  preference  is  preferred.  This  record 
takes  the  fonn  of  a  rating  scale,  where  pilots  are  able  to  quantify  their  subjective  opinion 
of  a  certain  aircraft.  The  unique  challenge  of  a  handling  qualities  evaluation  is  that  a 
pilot’s  opinions  are  not  used  as  engineering  data.  The  rating  scales  are  used  as  a  way  of 
summarizing  the  opinions  into  an  evaluation. 

2.2.1  Cooper-Harper  Rating  Scale 

Now  that  handling  qualities  have  been  defined  and  discussed,  how  can 
pilots  quantitatively  use  them  to  rate  aircraft  in  a  repeatable  manner?  HQ  rating  scales 
provide  the  answer  to  the  problem  but  there  are  several  to  choose  from.  “The  Cooper- 
Harper  scale  is  the  most  commonly  used  numerical  rating  scale”  in  flight  test,  and  “is 
universally  used  to  enable  the  pilot  to  award  a  number  to  an  aircraft  to  allow  comparison 
with  other  aircraft  or  to  show  compliance  with  a  specification”  (Hodgkinson,  1998).  It  is 
this  commonality  and  universal  acceptance  among  test  pilots  that  drove  the  decision  to 
use  the  Cooper-Harper  scale  throughout  the  course  of  this  study. 

When  George  E.  Cooper  from  the  National  Aeronautics  and  Space  Administration 
(NASA)  and  Robert  P.  Harper  Jr.  from  Cornell  University  combined  their  research  in 
1966,  the  Cooper-Harper  rating  (CHR)  scale  came  into  existence.  In  1970  it  was  adopted 
as  the  basis  of  the  US  flying  qualities  Military  Specification,  Mil-F-8785B.  The  Cooper- 
Harper  scale  has  ten  different  points,  where  a  10  represents  the  worst  HQ  possible  and  a  1 
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represents  excellent  HQ.  “The  scale  is  dichotomous,  which  improves  repeatability  by 
leading  the  evaluation  pilot  through  a  series  of  decisions  regarding  the  task  performance 
and  the  pilot  workload”  (Hodgkinson,  1998).  A  pilot’s  analysis  of  these  qualities  through 
the  rating  scale  is  then  used  to  either  evaluate  aircraft  operational  effectiveness  or  to 
match  favorable  characteristics  with  various  aircraft  configurations  in  an  effort  to 
improve  the  overall  design.  The  Cooper-Harper  scale  is  shown  in  Figure  1,  and  details 
the  various  pilot  decisions  made  throughout  the  course  of  a  test  evaluation.  Each 
decision  leads  to  a  yes  or  no  answer,  there  is  only  a  single  way  to  reach  a  certain  CHR 
level.  The  scale  describes  not  just  the  pilot’s  decision  tree,  but  whether  improvements  are 
necessary,  what  the  aircraft  characteristics  are  for  each  level,  and  how  that  corresponds  to 
pilot  workload.  The  scale  was  designed  for  its  repeatability  over  a  vast  set  of  test 
conditions. 
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Adequacy  for  Demand  on  pilot 

selected  task  or  Aircraft  in  selected  task  Pilot 

required  operation  charactenstcs  or  required  operation  rating 


Figure  1  -  Cooper  Harper  Rating  Scale 
2.2.2  Pilot  In-the-loop  Oscillation  (PIO)  Rating  Scale 

There  are  several  other  handling  qualities  scales  besides  the  Cooper- 
Harper  scale.  The  Pilot  In-the-Loop  Oscillation  (PIO)  rating  scale  is  depicted  in  figure  2. 
It  is  a  specialized  scale  directed  at  HQ  problems  that  are  known  or  suspected  to  cause 
PIOs.  This  scale  is  widely  used  in  the  test  world,  but  its  ratings  can  be  scattered  more 
than  Cooper-Harper  ratings,  due  mostly  to  the  difficulty  in  describing  if  a  PIO  is  an 
annoyance.  The  principle  purpose  of  this  scale  is  to  initiate  discussion  on  the  topic  of 
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PIOs  between  engineers  and  pilots.  Following  the  PIO  scale  figure  is  table  1,  which 
depicts  the  individual  rating  number  and  corresponding  description. 


Figure  2  -  PIO  Rating  Scale 
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Table  1  -  PIO  Ratings  and  Descriptions 


Description 

Rating 

No  tendency  for  pilot  to  induce  undesirable  motion 

1 

Undesirable  motions  tend  to  occur  when  pilot  initiates 
abrupt  maneuvers  or  attempts  tight  control.  These 
motions  can  be  prevented  or  eliminated  by  pilot 
technique. 

2 

Undesirable  motions  easily  induced  when  pilot 
initiates  abrupt  maneuvers  or  attempts  tight  control. 
These  motions  can  be  prevented  or  eliminated  but  only 
at  sacrifice  to  task  performance  or  through 
considerable  pilot  attention  and  effort. 

3 

Oscillations  tend  to  develop  when  pilot  initiates  abrupt 
maneuvers  or  attempts  tight  control.  Pilot  must  reduce 
gain  or  abandon  task  to  recover. 

4 

Divergent  oscillations  tend  to  develop  when  pilot 
initiates  abrupt  maneuvers  or  attempts  tight  control. 
Pilot  must  open-loop  by  releasing  or  freezing  the  stick. 

5 

Disturbance  of  normal  pilot  control  may  cause 
divergent  oscillation.  Pilot  must  open  control  loop  by 
releasing  or  freezing  the  stick. 

6 

2.2.3  Failure  Rating  Scale 

In  circumstances  where  the  Cooper-Harper  decision  matrix  is  not  straight 
forward  in  application  another  scale  may  be  required.  An  example  of  this  type  of  scale  is 
the  Failure  Rating  scale  (Hodgkinson,  1998),  shown  in  figure  3.  This  scale  was 
developed  at  NASA  Ames  for  evaluating  failures  and  recoveries. 
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Safe  operating  condition  =  Within  both  aircraft  and  operational  limits 

Recovery  =  Return  to  safe  operating  condition 

Effort  =  Integrated  physical  and  mental  workload  required  to  execute 

recovery 

Compromise  safety  of  flight  =  Cause  to  exceed  either  aircraft  or 
operational  limits  or  cause  an  encounter  with  surface  obstacles 

Figure  3  -  Failure  Rating  Scale 


2.2.4  Other  Pilot  Rating  Scales 

The  particular  objectives  of  a  handling  qualities  evaluation  will  vary,  and 
the  research  may  need  to  incorporate  other  ratings  scales  that  are  more  useful  when 
conducting  a  specific  type  of  testing.  Some  other  examples  of  pilot  rating  scales  are 
displayed  in  figures  4-6  to  show  just  a  few  of  the  many  choices  available  for  rating  the 
handling  qualities  of  an  aircraft.  Figure  4  shows  two  Useable  Cueing  Environment  scales 


18 


(Hodgkinson,  1998).  These  scales  are  designed  to  measure  the  level  of  pilot-vehicle 
interaction.  An  example  of  this  is  synthetic  vision  use  in  a  cockpit,  where  these  scales 
can  measure  how  well  necessary  cueing  information  is  provided  to  the  pilot,  and  how  that 
information  impacts  the  pilot’s  assessment. 


1  - 

-  Good 

1  - 

-  Good 

1  - 

-  Good 

2  - 

2  - 

2  - 

3  - 

-  Fair 

3  - 

-  Fair 

3  - 

-  Fair 

4  - 

4  - 

4  - 

5  - 

-  Poor 

5  - 

-  Poor 

5  - 

-  Poor 

Attitude 

Horizontal 

Translational 

Rate 

Vertical 

Translational 

Rate 

Pitch,  roll  and  yaw  attitude,  and  lateral-longitudinal,  and 
vertical  translational  rates  shall  be  evaluated  for  stabilization 
effectiveness  according  to  the  following  definitions: 

Good  :  Can  make  aggressive  and  precise  corrections 
with  confidence  and  precision  is  good. 

Fair :  Can  make  limited  corrections  with  confidence 
and  precision  is  only  fair. 

Poor:  Only  small  and  gentle  corrections  are  possible, 
and  consistent  precision  is  not  attainable. 


Attitude  VCR 


Figure  4  -  Useable  Cueing  Environment  Scales 
The  rating  scale  in  figure  5  shows  the  level  of  situational  awareness  of  the  pilot 
while  performing  certain  tasks  (Hodgkinson,  1998).  Situational  awareness  is  the  concept 


of  being  able  to  observe  the  present  and  remember  the  past  in  order  to  predict  the  future. 


Absolute  situational  awareness  would  allow  an  individual  to  observe  and  understand 


everything  in  their  surroundings,  correlate  that  information  with  events  that  have  already 
occurred,  and  then  make  a  conclusion  about  what  will  subsequently  take  place.  A  general 
rule  is  that  as  pilot  workload  increases,  situational  awareness  decreases.  This  scale  can 
be  used  to  actually  rate  situational  awareness,  or  can  be  used  in  combination  with  another 
rating  scale  in  an  effort  to  measure  pilot  workload. 
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China  Lake  SA  Rating  Scale 


SA  SCALE 


VALUE 

CONTENT 

VERY  GOOD  • 

1  • 

Full  knowledge  of  a/c  energy  state/tactical  environment/mission; 

Full  ability  to  anticipate/accommodate  trends 

GOOD 

2  • 

• 

Full  knowledge  of  a/c  energy  state/tactical  environment/mission; 

Partial  ability  to  anticipate/accommodate  trends; 

No  task  shedding 

ADEQUATE  • 

3  • 

• 

Full  knowledge  of  a/c  energy  state/tactical  environment/mission; 

Saturated  ability  to  anticipate/accommodate  trends; 

Some  shedding  of  minor  tasks 

POOR  • 

4  • 

• 

Fair  knowledge  of  a/c  energy  state/tactical  environment/mission; 

Saturated  ability  to  anticipate/accommodate  trends; 

Shedding  of  all  minor  tasks  as  well  as  many  not  essential  to  flight  safety/mission 
effectiveness 

VERY  POOR  • 

5  • 

• 

Minimal  knowlege  of  a/c  energy  state/tactical  environment/mission; 

Oversaturated  ability  to  anticipate/accommodate  trends; 

Shedding  of  all  tasks  not  absolutely  essential  to  flight  safety/mission  effectiveness 

Figure  5  -  China  Lake  Situational  Awareness  Scale 

Another  pilot  rating  scale  is  the  USAFAM  Workload  scale  of  figure  6 
(Hodgkinson,  1998).  As  the  name  suggests,  it  is  a  measure  of  the  pilot  workload,  or  how 
highly  tasked  the  pilot  feels  when  trying  to  accomplish  a  certain  scenario.  As  pilot 
workload  increases,  mission  effectiveness  will  begin  to  decrease,  and  if  raised  to  a  high 
enough  level,  will  impact  the  ability  to  maintain  flight. 
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1 

NOTHING  TO  DO; 

No  system  demands 

2 

LIGHT  ACTIVITY; 

Minimum  system  demands 

3 

MODERATE  ACTIVITY; 

Easily  managed; 
Considerable  spare  time 

4 

BUSY; 

Challenging  but  manageable; 
Adequate  time  available 

5 

VERY  BUSY; 

Demanding  to  manage; 

Barely  enough  time 

6 

EXTREMELY  BUSY; 

Very  difficult; 

Nonessential  tasks  postponed 

7 

OVERLOADED; 

System  unmanageable; 
Essential  tasks  undone;  Unsafe 

Figure  6  -  USAFAM  Workload  Scale 


2.3  Next-Generation  Long-Range  Strike  Aircraft 

This  thesis  investigates  a  concept  aircraft  designed  to  meet  the  Air  Force 
requirement  for  a  new  long-range  strike  capability.  This  section  explains  the  impetus  for 
the  new  aircraft  and  one  defense  contractor’s  efforts  to  design  it.  Although  several 
contractors  generated  designs  in  response  to  the  new  requirement,  this  thesis  looks  only  at 
one  design  concept,  Northrop  Grumman  Corporation’s  Supersonic  Tailless  Air  Vehicle 
(STAV). 

2.3.1  2006  Quadrennial  Defense  Review 

The  US  Department  of  Defense  (DoD)  conducts  a  review  of  its  vision  and 
mission  every  four  years  in  order  to  better  focus  its  efforts  in  a  rapidly  changing  world. 
A  result  of  this  work  was  the  2006  Quadrennial  Defense  Review  (QDR)  report.  The 
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2006  QDR  was  especially  poignant,  because  it  was  the  first  QDR  conducted  in  the  post- 
September  11th  world,  while  the  nation  was  at  war.  It  details  the  manner  in  which  the 
DoD  will  fight  the  “Long  War”.  It  states:  “Joint  air  capabilities  must  be  reoriented  to 
favor  systems  that  have  far  greater  range  and  persistence;  larger  and  more  flexible 
payloads  for  surveillance  or  strike;  and  the  ability  to  penetrate  and  sustain  operations  in 
denied  areas.  The  future  force  will  exploit  stealth  when  and  where  it  is  needed.  The  Air 
Force  has  set  a  goal  of  increasing  its  long-range  strike  capabilities  by  50%  and  the 
penetrating  component  of  long-range  strike  by  a  factor  of  five  by  2025.  Approximately 
45%  of  the  future  long-range  strike  force  will  be  unmanned.  The  capacity  for  joint  air 
forces  to  conduct  global  conventional  strikes  against  time-sensitive  targets  will  also  be 
increased.  To  achieve  the  future  joint  force  characteristics,  the  DoD  plans  to  develop  a 
new  land-based,  penetrating  long-range  strike  capability  to  be  fielded  by  2018”  (QDR, 
2006). 

2.3.2  Northrop  Grumman  Corporation  Design  Program 

In  response  to  the  2006  QDR,  the  Northrop  Grumman  Corporation  (NGC) 
began  a  design  program  to  meet  the  needs  of  the  USAF.  This  program  included  several 
different  concepts,  including  a  long-range  strike  (LRS)  aircraft  and  two  regional 
bombers.  These  aircraft  were  designed  to  meet  all  mission  threshold  range  and  speed 
goals  set  by  the  Air  Force.  This  resulted  in  design  concepts  that  differed  from 
conventional  strike  aircraft  in  several  ways.  First,  to  meet  stealth  and  speed 
requirements,  these  supersonic  aircraft  had  no  tails.  Second,  the  cockpit  location  was 
well  aft  of  a  standard  cockpit  location  for  structural  reasons  designed  to  reduce  drag. 
Third,  driven  by  the  stealth  requirement,  crew  visibility  out  of  the  cockpit  was  extremely 
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limited,  meaning  that  most,  if  not  all,  of  the  pilot  visibility  outside  the  cockpit  would  have 
to  be  synthetic.  Finally,  the  instantaneous  center  of  rotation  (ICR)  of  the  aircraft  was 
located  far  forward  of  a  conventional  aircraft’s  center  of  rotation.  Rather  than  being 
located  near  the  center  of  gravity  (CG),  the  ICR  was  thirty  feet  in  front  of  the  CG,  almost 
collocated  with  the  cockpit.  This  meant  that  the  initial  flight  path  response  to  a  given 
pitch  input  would  be  opposite  the  direction  of  the  input.  This  response  would  be  most 
pronounced  to  the  pilot  during  approach  and  landing,  where  an  input  to  climb  would 
initially  result  in  motion  towards  the  ground.  Furthermore,  the  sink  rate  perceived  by  the 
pilot  in  the  cockpit  would  be  much  less  than  the  actual  sink  rate  of  the  landing  gear, 
resulting  in  a  potentially  dangerous  rate  of  descent. 

All  of  these  non-conventional  design  aspects  combined  to  form  an  aircraft  with  a 
supersonic  tailless  delta  configuration.  Figure  7  shows  an  artist’s  rendering  of  a  potential 
Supersonic  Tailless  Air  Vehicle  (STAV).  Such  vehicles  are  known  to  be 
aerodynamically  complex  aircraft  with  distinctive  flight  dynamic  characteristics  and 
intricate  flight  control  laws.  Therefore,  a  handling  qualities  evaluation  of  this  aircraft 
was  important  to  ensure  that  the  aircraft  control  laws  and  flight  control  system  had  been 
properly  designed  and  modeled. 
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Figure  7  -  Supersonic  Tailless  Air  Vehicle 

The  Northrop  Grumman  Corporation  came  up  with  several  different  flight  control 
suites  to  use  in  the  various  concept  aircraft  (Northrop  Grumman,  2007).  The  control 
suites  consisted  of  control  effector  layouts  of  different  size  and  type.  A  study  on  the 
stability,  control,  and  aero-performance  of  high  lift-to-drag  ratio  supersonic  tailless  air 
vehicles  was  conducted  to  determine  which  suites  met  requirements  and  provided  the  best 
aerodynamic  and  survivability  solutions.  Wind-tunnel  tests  were  conducted  to  ensure 
that  the  different  control  effector  layouts  and  flight  control  laws  were  being  properly 
modeled  and  to  aid  in  the  creation  of  pilot-in-the-loop  fixed-base  simulations.  These 
fixed-base  simulations  were  then  used  to  update  the  models,  flight  control  laws,  and 
address  any  control  interference  or  power  deficiencies.  Three  NGC  test  pilots  and  two 
USAF  pilots  conducted  over  fifteen  hours  of  evaluations  of  control  power  gains  and 
piloting  techniques.  These  techniques  included  both  front  and  back  side  of  the  power 
curve  piloting  techniques.  This  allowed  the  selection  of  the  control  effector  suite  that 
could  be  used  in  the  entire  STAV  flight  envelope.  Full-motion  simulations  could  then  be 
used  to  explore  control  law  design  and  different  control  effectiveness  challenges.  In 
order  to  accomplish  this  and  to  conduct  a  STAV/  LRS  handling  qualities  assessment, 
NGC  combined  with  Air  Force  Research  Laboratory’s  Air  Vehicles  Directorate 


24 


(AFRL/RBCD)  to  conduct  these  full-motion  simulations  on  the  five  degree-of-freedom 


Large  Amplitude  Multi-mode  Aerospace  Research  Simulator  (LAMARS).  Figures  8-11 


depict  the  heads  down  display,  attitude  direction  indicator,  and  heads  up  display  (HUD) 


used  during  the  test  simulations,  as  well  as  the  LAMARS  itself. 
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Figure  8  -  Heads  Down  Display 


Figure  9  -  Attitude  Direction  Indicator 
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Figure  11  -  LAMARS  at  Air  Force  Research  Laboratory 
This  full-motion  testing  involved  five  pilots  (including  the  author)  flying  over  400 
simulations  runs  covering  forty-eight  different  test  scenarios  at  Mach  2+  supersonic, 
subsonic  up-and-away,  and  powered-approach  and  landing  flight  conditions.  Testing 
revealed  that  the  control  laws  and  aerodynamic  effectiveness  of  the  control  surfaces  were 
stressed  the  most  during  the  low-speed  approach  and  landing  test  conditions.  Test 
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scenarios  were  also  completed  using  “synthetic  vision”  displays  in  place  of  the  “out-the- 
window”  cockpit  view.  The  following  table  2  displays  the  various  test  scenarios,  defined 
as  tasks,  which  included  tracking  and  precision  landing  tasks  in  visual  flight  rules, 
instrument  flight  rules,  and  crosswind  conditions. 

Table  2  -  Initial  LAMARS  Test  Scenarios 


Task 

Task  Name 

J  Approach  and  Landing  Tasks 

i 

Nominal  ILS  Approach 

2 

Precision  Landing 

3 

Lateral  Offset  Landing 

4 

Vertical  Offset  Landing 

5 

Go-Around 

|  Low  Altitude  Cruise  Tasks 

i 

Attitude  Capture  (Theta)  in  Low  Altitude  Cruise 

2 

Heading  Change  in  Low  Altitude  Cruise  1 

3 

Steady  Heading  Sideslip  in  Low  Altitude  Cruise 

j  Supersonic  Cruise  Tasks 

i 

Altitude  Capture  in  Supersonic  Cruise 

2 

Attitude  Capture  (Theta)  in  Supersonic  Cruise  | 

3 

Heading  Change  in  Supersonic  Cruise  i 

4 

Steady  Heading  Sideslip  in  Supersonic  Cruise 

|  Synthetic  Vision  Tasks 

i 

Nominal  Synthetic  Vision  ILS  Approach  ; 

2 

Lateral  Offset  Landing  with  Synthetic  Vision  1 

The  pilot  ratings  and  comments  from  this  battery  of  tests  showed  that  STAV  was 
not  yet  a  “level  one  or  level  two”  aircraft  in  most  flying  conditions.  The  pilots 
experienced  some  non-minimum  phase  behavior  in  pitch  and  yaw,  which  could  result  in  a 
Pilot  In-the-loop  Oscillation  (PIO)  prone  aircraft.  However,  the  NGC  concluded  that 
with  improvements  to  the  control  laws  coupled  with  additional  aids  to  alleviate  pilot 
workload,  these  ratings  could  improve.  It  was  apparent  to  them  that  the  existing  control 
effector  suite  was  likely  capable  of  delivering  “level  one  or  high  level  two”  handling 
qualities  throughout  the  STAV  flight  envelope.  Further  testing  would  include  STAV 
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simulations  with  an  improved  flight  control  model  that  included  ground  effect  in  the  low 
speed  flight  regime. 

2.4  Related  Research 

Two  other  sources  of  research  that  involve  the  testing  and  handling  qualities 
evaluations  of  supersonic  semi -tailless  air  vehicles  are  the  F-16XL  program  and  the 
Space  Shuttle  Orbiter  program.  Both  of  these  aircraft  differ  from  the  STAV,  because 
they  each  possess  a  large  vertical  tail.  However,  they  are  similar  to  the  STAV  due  to  their 
lack  of  any  horizontal  tail  and  semi-delta  wing  configuration.  Of  particular  interest  is  the 
Space  Shuttle  Orbiter’s  instantaneous  center  of  rotation,  which  like  the  STAV  is  located 
far  forward  of  a  conventional  center  of  rotation. 

2.4.1  F-16  XL 

The  F-16XL  program  began  in  the  early  1980’s  when  two  F-16s  were 
modified  by  extending  their  fuselage  length  and  incorporating  a  large  area  delta  wing 
planform.  What  started  as  a  derivative  fighter  evaluation  program  turned  into  an  ability 
to  test  concepts  in  support  of  future  high-speed  supersonic  transport  aircraft  (Stachowiak, 
2004).  This  included  an  attempt  to  reduce  drag  by  achieving  natural  laminar  flow 
through  careful  contouring  of  the  wing  surface  and  active  laminar  flow  control  using  an 
internal  suction  system  built  into  the  wing  (Anderson,  1992).  In  order  to  expand  the 
capabilities  of  this  test  platfonn,  the  aircraft  was  updated  with  a  digital  flight  control 
system  (DFLCS).  A  handling  qualities  analysis  of  the  F-16XL  with  DLFCS  incorporated 
commenced  in  December  of  1997.  Throughout  the  course  of  ten  test  flights,  the  Cooper- 
Harper  HQ  rating  scale  was  applied  to  collected  flight  data,  and  compared  with 
qualitative  pilot  assessments  of  the  HQ  (Stachowiak,  2004).  The  flight  tests  included 
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various  handling  qualities  tasks:  normal  acceleration,  pitch  attitude,  and  bank  angle 
captures,  air-to-air  tracking,  close  trail  formation  flight,  and  powered  approach.  Cooper- 
Harper  HQ  assessments  were  made  for  each  task,  from  both  a  gross  acquisition  and  fine 
tracking  standpoint.  A  picture  of  the  F-16XL  taken  during  flight-testing  is  depicted  in 
Figure  12. 


F-16XL  ship  #1  (#849)  with  Digital  Flight  Control  System  (DFCS)  in  flight  over  desert 

Figure  12  -  F-16XL  in  Flight 

2.4.2  Space  Shuttle  Orbiter/  Total  In-Flight  Simulator  (TIFS) 

The  Space  Shuttle  Orbiter  program  began  in  the  1970’s  and  has  been 
constantly  tested  throughout  its  life.  It  was  designed  to  return  to  earth  not  via  parachute 
as  the  Mercury,  Gemini,  and  Apollo  programs  had,  but  by  gliding  back  to  land  on  a 
runway.  The  shuttle  was  equipped  with  an  automatic  landing  program,  because  the  space 
shuttle  (like  other  gliders),  only  had  one  chance  to  land  during  recovery  from  space.  The 
designers  then  set  about  attempting  to  provide  a  manual  landing  capability  for  an 
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operational  crew  and  vehicle  returning  from  orbit,  just  in  case  the  automatic  landing 
program  was  not  working.  Manually  landing  the  orbiter  in  an  operational  environment 
proved  exceedingly  complex  to  accomplish,  particularly  due  to  the  longitudinal  handling 
qualities  of  the  vehicle.  This  was  due  in  large  part  to  the  fact  that  the  instantaneous 
center  of  rotation  of  the  orbiter  was  located  in  front  of  the  actual  vehicle.  This  provided  a 
non-minimum  phase  response,  where  the  flight  path  initially  moved  opposite  to  a  given 
longitudinal  input,  a  flight  characteristic  that  caused  concern  for  an  un-powered  aircraft 
close  to  the  ground.  Test  flights  of  the  orbiter  involving  approach  and  landing  tasks 
indicated  a  tendency  to  PIO  near  landing,  as  demonstrated  in  a  1977  test  landing  before 
the  Prince  of  Wales  at  Edwards  AFB.  The  task  of  landing  was  made  easier  by  changing 
the  operational  procedures,  and  an  adaptive  stick  filter  was  employed  on  the  orbiter  to 
reduce  the  magnitude  of  any  encountered  PIOs.  The  evaluations  included  tests  in  fixed- 
base,  full-motion,  and  in-flight  simulators.  The  orbiter  control  system  and  procedures 
provided  satisfactory  performance  in  conducting  precision  landings  with  a  large,  low  lift- 
to-drag  ratio  glider.  Figure  13  shows  the  space  shuttle  Atlantis  landing  after  a  mission  in 
1988. 
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Figure  13  -  Space  Shuttle  Atlantis  Landing 

In  the  mid- 1 980 ’s,  a  new  control  system  designed  to  improve  the  orbiter 
longitudinal  response  characteristics  was  investigated.  This  system  improved  the  orbiter 
flight  path  response  by  increasing  the  amount  of  pitch  rate  overshoot  and  reducing  the 
overall  time  delay.  The  NASA  Ames-Dryden  Flight  Research  Facility  conducted  test 
simulations  of  the  shuttle  during  landing  using  the  Ames  Research  Center  vertical  motion 
simulator  and  the  Total  In-Flight  Simulator  (TIFS)  variable-stability  aircraft.  During 
these  tests,  it  became  evident  that  pilot  background  characteristics  were  influencing  their 
opinion  of  the  new  orbiter’ s  HQ  rating.  Trained  and  experienced  astronauts  who  were 
familiar  with  the  old  control  system  found  the  new  system  to  be  inferior,  while  pilots 
without  extensive  training  or  experience  on  the  shuttle  strongly  preferred  the  new  system. 
The  cause  of  this  difference  in  rating  was  hypothesized  to  be  the  different  control 
strategies  of  the  two  pilot  groups.  These  control  strategies  were  interpreted  in  terms  of 
open-loop  aircraft  response  characteristics  and  pilot-vehicle  closed-loop  characteristics 
(Powers,  1986). 
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2.5  Impacts  on  Handling  Qualities  during  Powered  Approach 

The  pilot’s  perception  of  an  aircraft’s  handling  qualities  is  impacted  by  numerous 
factors.  The  handling  qualities  can  be  more  accurately  evaluated  if  the  pilot  is  able  to 
discern  why  the  aircraft  responds  in  a  certain  manner.  This  section  details  concepts 
particularly  important  when  evaluating  a  highly-augmented  aircraft  in  the  approach  and 
landing  environment. 

2.5.1  Longitudinal  Stability 

Before  discussing  longitudinal  stability  specifically,  it  is  important  to 
define  several  terms  that  will  be  used  in  the  discussion.  The  first  is  angle  of  attack,  a, 
which  is  the  angle  made  between  the  body-fixed  axis  pointing  out  the  nose  of  the  aircraft 
and  the  tangent  to  the  flight  path  at  the  aircraft  center  of  gravity.  The  flight  path  angle,  y, 
is  the  angle  made  between  the  velocity  vector  of  the  aircraft  center  of  gravity  and  the 
horizon.  Finally,  the  pitch  angle,  9,  is  the  angle  made  between  the  body-fixed  axis 
pointing  out  the  nose  of  the  aircraft  and  the  horizon.  Pitch  and  flight  path  angles  both 
reference  the  earth  and  are  inertial,  while  angle  of  attack  can  be  determined  using  the 
relation  a  =  9  -  y.  Illustrations  of  all  three  angles  can  be  found  in  Nelson  (Nelson,  1998). 

The  tendency  of  an  aircraft  to  return  to  pitch  equilibrium  after  encountering  a 
disturbance  in  angle  of  attack  is  the  definition  of  static  longitudinal  stability.  If  a 
disturbance  pitches  the  aircraft  nose  up,  then  a  longitudinally  statically  stable  aircraft  will 
produce  a  nose-down  pitching  moment.  For  static  stability,  this  means  that  the  moment 
coefficient  due  to  angle  of  attack,  Cma,  is  negative.  If  the  pitching  moment  is  zero,  then 
the  aircraft  is  longitudinally  trimmed.  In  order  to  fly,  a  conventional  aircraft  must  trim  at 
a  positive  angle  of  attack  and  be  longitudinally  statically  stable.  This  is  normally  done 
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using  the  elevator  on  the  aircraft  tail.  A  tailless  aircraft  needs  to  use  complex  flight 
control  effectors  to  maintain  static  longitudinal  stability. 

The  neutral  point  of  an  aircraft  is  its  aerodynamic  center,  the  point  at  which  the 
pitching  moment  is  constant  when  angle  of  attack  is  varied.  The  static  margin  of  an 
aircraft  is  the  distance  the  center  of  gravity  is  in  front  of  the  aerodynamic  center,  and  is 
directly  proportional  to  Cma  (Hodgkinson,  1998).  If  the  center  of  gravity  is  moved  too 
far  aft  towards  the  aerodynamic  center,  then  the  aircraft  will  become  longitudinally 
statically  unstable.  The  final  part  of  longitudinal  static  stability  is  speed  stability.  Stick- 
fixed  speed  stability  is  positive  if  larger  and  larger  longitudinal  nose-down  stick 
deflections  are  required  as  the  trim  airspeed  is  increased.  Stick-free  speed  stability  is 
positive  if  larger  and  larger  longitudinal  nose-down  stick  forces  are  required  as  the  trim 
airspeed  is  increased. 

The  dynamic  longitudinal  stability  of  an  aircraft  involves  two  main  factors,  the 
phugoid  and  short  period  flying  modes  of  motion.  In  a  conventional  aircraft,  these  two 
modes  are  a  good  indication  of  what  the  handling  qualities  will  be.  The  phugoid  mode  is 
a  low-frequency  motion  that  interchanges  altitude  and  airspeed.  It  causes  altitude, 
airspeed,  pitch,  and  flight  path  oscillations  while  maintaining  a  nearly  constant  angle  of 
attack.  The  magnitude  of  this  motion  is  small,  and  is  usually  controlled  simply  by  the 
pilot’s  nonnal  pitch  inputs.  However,  if  poor  phugoid  characteristics  are  present,  more 
pilot  compensation  will  be  required,  making  non-flying  tasks  more  difficult  to  complete. 
The  following  table  shows  phugoid  damping  ratio,  £p,  requirements  (MIL  STD  1797B, 
2006)  and  how  they  relate  to  handling  qualities  levels.  The  time-to-double  amplitude, 
Tq2,  is  the  time  it  takes  for  the  magnitude  of  the  phugoid  motion  to  double  in  size. 
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Table  3  -  Phugoid  Damping  Ratio  Requirements 


Handling  Qualities  Level 

Phugoid  Damping  Ratio  Required 

1 

>  0.04 

2 

Cd  >  0.0 

3 

Tq2  >  55  seconds 

The  longitudinal  mode  of  motion  with  the  greatest  impact  on  handling  qualities 
rating  is  the  short  period  mode.  The  short  period  governs  the  transient  response  of  pitch, 
flight  path,  and  angle  of  attack  to  a  rapid  control  input  or  wind  gust  (Hodgkinson,  1998). 
During  the  short  period  oscillations,  which  are  generally  under-damped  and  stable,  the 
airspeed  and  flight  path  remain  nearly  constant  while  the  pitch  and  angle  of  attack  vary. 
Although  the  duration  of  the  motion  is  brief,  it  has  a  significant  impact  on  the  handling 
qualities  rating.  The  following  table  shows  the  required  short  period  damping  ratio,  Csp, 
for  different  handling  qualities  levels  and  flight  phase  categories  (MIL  STD  1797B, 
2006). 


Table  4  -  Short  Period  Damping  Ratio  Requirements 


Category  A  and  C  Flight  Phases 

Category  B 

Flight  Phases 

HQ  Level 

Minimum  £SIJ 

Maximum  £SD 

Minimum  CSD 

Maximum  £sd 

1 

0.35 

1.30 

0.30 

2.00 

2 

0.25 

2.00 

0.20 

2.00 

3 

0.15 

- 

0.15 

- 

A  pilot’s  ability  to  control  the  short  period  depends  not  only  upon  the  mode  itself, 
but  also  on  the  pitch  response  of  the  aircraft.  The  Control  Anticipation  Parameter  (CAP), 
takes  into  account  both  of  these.  It  is  defined  as  the  ratio  of  the  initial  pitch  acceleration 
to  the  final  normal  acceleration.  In  an  aircraft  with  good  CAP,  the  initial  and  final 
accelerations  perceived  by  the  pilot  will  match  the  pilot’s  expectations.  CAP  is 
proportional  to  the  square  of  the  short  period  frequency,  cosp.  The  CAP  and  Csp  can  be 
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plotted  against  one  another  to  predict  the  aircraft  handling  qualities  levels.  There  are 
several  other  concepts  that  involve  longitudinal  dynamic  stability  in  feedback  control 
systems,  including:  equivalent  systems  and  time  delay,  bandwidth  and  Neal-Smith 
methods,  and  drop-back  criterion.  Hodgkinson  (Hodgkinson,  1998)  delves  into  further 
discussion  of  these  topics. 

2.5.2  Lateral/  Directional  Stability 

Unlike  longitudinal  motions,  which  can  be  considered  two-dimensional, 
lateral/  directional  motion  is  usually  seen  as  more  complex.  This  arena  involves  roll, 
yaw,  and  side  translation  degrees  of  freedom.  In  order  to  simplify  the  discussion  of 
lateral/  directional  stability,  the  angles  of  roll,  yaw,  and  sideslip  require  explanation.  The 
roll  angle,  (j),  is  the  angle  made  between  the  axis  out  the  right  wing  of  the  aircraft  and  the 
horizontal,  and  is  considered  positive  when  the  right  wing  is  down.  The  yaw  angle,  \|/,  is 
the  angle  between  the  axis  pointing  out  the  nose  of  the  aircraft  and  an  arbitrary  reference 
azimuth  line,  and  is  considered  positive  as  the  nose  moves  right.  The  sideslip  angle,  P,  is 
the  angle  made  between  the  aircraft  plane  of  symmetry  and  the  relative  wind  flow 
direction.  If  this  incident  flow  is  encountering  the  right  side  of  the  aircraft,  then  sideslip 
is  considered  positive.  Illustration  of  these  angles  can  again  be  found  in  Nelson  (Nelson, 
1998). 

If  the  aircraft  response  to  an  increase  in  sideslip  angle  is  a  restoring  moment 
putting  the  aircraft  nose  into  the  relative  wind,  then  that  aircraft  is  directionally  statically 
stable  (positive  Cnp).  If  the  aircraft  response  to  a  nose-right  sideslip  is  a  left  wing-down 
roll,  then  that  aircraft  is  laterally  statically  stable,  and  is  said  to  have  positive  dihedral 
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(negative  Qp).  Lateral/  directional  static  stability  is  simply  a  steady-state  case  of 
dynamic  stability. 

Lateral/  directional  dynamic  stability  includes  three  different  modes  of  motion, 
the  roll,  Dutch  roll,  and  spiral  modes.  The  roll  mode  depends  highly  on  the  taper  and 
aspect  ratios  of  the  aircraft  wing.  When  an  aircraft  rolls,  the  roll  inertia  induces  a 
resisting  moment  that  is  proportional  to  the  product  of  the  roll  acceleration  and  the  roll 
inertia  itself.  This  resisting,  damping  moment  is  caused  mainly  by  the  aircraft  wing, 
because  the  down-going  wing  experiences  a  higher  angle  of  attack  and  higher  lift, 
resulting  in  the  opposing  moment  to  the  roll.  This  roll  mode  time  constant  is  around  a 
second  for  fighter-type  aircraft,  any  longer  than  this  and  the  pilot  would  feel  as  if  they 
were  commanding  roll  acceleration  rather  than  rate  (Hodgkinson,  1998).  The  spiral  mode 
is  best  described  as  a  slow  divergence  from  a  disturbance  in  roll  angle.  If  allowed  to 
continue,  a  slightly  unstable  spiral  mode  would  cause  an  aircraft  to  slowly  spiral  in  a 
descending,  turning  motion.  This  motion  is  generally  benign  and  easy  for  the  pilot  to 
control,  and  can  be  slightly  unstable  yet  still  allow  level  one  handling  qualities.  The  final 
mode  is  that  of  Dutch  roll,  an  oscillatory  short-period  motion  in  roll  and  yaw.  It  is 
considered  by  pilots  to  be  an  annoying  motion  experienced  in  the  roll  or  yaw  response  to 
a  lateral  or  directional  control  input. 

The  lateral/  directional  handling  qualities  rating  given  by  the  pilot  is  impacted  by 
the  roll  angle  to  sideslip,  or  (j)/  p  ratio.  This  ratio  can  be  used  to  predict  some  of  the 
lateral/  directional  problems  that  a  pilot  might  experience  while  in  flight.  If  the  ratio  is 
low  (less  than  one),  then  the  Dutch  roll  cannot  be  damped  with  lateral  control,  and  roll 
maneuvers  will  be  imprecise  due  to  lateral  nose  motion.  If  the  ratio  is  medium  (one  to 
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two),  then  roll  precision  can  be  adversely  effected  by  roll  oscillations  or  “ratcheting” 
(Hodgkinson,  1998).  If  the  ratio  is  high  (greater  than  two),  then  unwanted  roll 
oscillations  may  be  caused  by  turbulence  or  rudder  inputs.  The  handling  qualities  of  a 
given  aircraft  depend  upon  both  static  and  dynamic  longitudinal  and  lateral  stability 
factors. 

2.5.3  Dynamic  Inversion 

An  aircraft’s  flight  control  system  can  significantly  impact  its  HQ.  One 
flight  control  method  used  by  NGC  on  its  STAV  flight  control  system  will  be  briefly 
discussed.  NGC  used  a  modern  aircraft  control  theory  called  dynamic  inversion  as  part 
of  their  design  for  the  STAV  flight  control  system.  With  dynamic  inversion,  a  specific 
set  of  desired  dynamics  is  used  to  replace  the  existing,  undesirable  dynamics.  It  can  be 
used  for  either  non-linear,  single-input-single-output  or  multiple-input-multiple-output 
systems,  provided  that  the  respective  control  effectiveness  function  or  control  influence 
matrix  is  invertible  (Shankar,  2003). 

The  dynamic  inversion  technique  inverts  the  dynamic  equations  of  the  aircraft 
plant  in  an  effort  to  specify  the  desired  plant  behavior.  It  accomplishes  this  explicitly  by 
stipulating  the  rate  of  the  control  variable,  rather  than  the  control  variable  itself,  where 
the  control  variable  refers  to  the  aircraft  state  being  controlled  (e.g.  angle  of  attack).  The 
undesired  dynamics  are  cancelled  and  replaced  algebraically  using  detailed  selection  of 
the  feedback  function.  Dynamic  inversion  is  also  known  as  feedback  linearization  based 
on  this  process.  The  key  assumption  of  this  control  theory  is  that  the  aircraft  plant 
dynamics  can  be  modeled  well,  and  can  therefore  be  cancelled  out  completely.  If  the 
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plant  cannot  be  modeled  well,  then  the  new  aircraft  dynamics  will  require  a  robust 
controller  to  suppress  any  undesired  dynamic  behavior. 

The  ultimate  goal  of  dynamic  inversion  is  to  find  a  controller  such  that  the  control 
variable  will  behave  as  desired.  Consider  the  following  example  of  dynamic  inversion  in 
flight  control.  The  non-linear  six  degree-of-freedom  model  is  described  by  the  function 
x  =  f(x,u),  where  the  states  are  defined  as  x  and  the  control  inputs  are  defined  as  u.  The 
control  variable  is  the  variable  to  be  controlled  and  is  a  nonlinear  function  of  the  state, 
CV  =  h(x),  where  h(x)  is  a  scalar  function.  The  control  variable  rate  can  then  be  defined 


as  cv=^cv  =  ±h(x) f(x,u) . 
dt  dt  ox  dt  ox  dx 


Then,  setting 


Oh(x^) 

f2(x,u)  = - — f(x,u )  =  CV  ,  the  control  law  u  =  g(x,  CV )  can  be  obtained  by  solving 

dx 


for  u  in  the  nonlinear  equation  f2(x,u)  =  CV  .  The  control  variable  rate,  CV  ,  is  then  set 


equal  to  the  desired  rate  ,  C Vdesired  ,that  ensures  the  desired  CV  response.  Assuming  the 
state  can  be  measured,  so  x  =  xmeaSured ,  the  commanded  input  is  given  by 
^commend  =  g(x measured ’ CV desired) .  A  more  comprehensive  discussion  of  this  topic  can  be 
found  in  the  Honeywell  report  (Honeywell,  1996). 

2.5.4  Center  of  Rotation 

An  aircraft’s  center  of  rotation  is  the  point  on  an  aircraft  about  which  all 
moments  or  rotations  take  place.  The  typical  location  of  this  point  corresponds  with  the 
vehicle’s  center  of  gravity,  the  point  which  represents  the  average  location  of  the  mass  of 
the  aircraft.  The  rotations  about  this  point  include  those  in  each  of  the  three  dimensions 
of  pitch,  roll,  and  yaw.  In  a  conventional  aircraft  design,  the  pilot  is  located  forward  of 
the  center  of  gravity  and  thus  the  center  of  rotation.  Given  a  command  by  the  pilot,  the 
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initial  flight  path  response  is  in  the  same  direction  as  the  long-term  aircraft  response. 
When  the  pilot  commands  a  pitch-up,  the  aircraft  will  respond  by  pitching  its  nose  up.  In 
these  cases,  the  instantaneous  center  of  rotation  (ICR)  closely  matches  the  overall  center 
of  rotation.  The  following  figure  illustrates  the  center  of  gravity  location  and  different 
axes  of  rotation. 


Figure  14  -  Aircraft  Body  Axes  and  Rotations 

In  more  advanced,  unconventional  aircraft  with  multiple  control  surface  locations, 
the  aircraft’s  ICR  can  be  placed  using  blending  of  the  control  surfaces  (Field,  2002).  If 
the  pitch  ICR  is  placed  in  front  of  the  center  of  gravity  (CG),  the  initial  flight  path 
response  at  the  center  of  gravity  will  be  in  the  opposite  direction  of  the  long-term  flight 
path  response.  The  greater  the  distance  between  the  aircraft’s  ICR  and  CG,  the  larger  the 
disparity  between  the  two  responses  will  become.  This  difference  in  response  is  most 
pronounced  to  pilots  during  demanding  phases  of  flight,  such  as  powered  approach  and 
landing  situations. 


39 


The  ability  to  make  consistent  landings  near  the  desired  touchdown  point  is  the 
pilot’s  main  objective  during  approach  and  landing.  In  order  to  accomplish  this,  the  pilot 
must  be  able  to  predict  and  accurately  control  the  main  gear’s  sink  rate  during  the  landing 
flare  (Field,  2002).  The  pilot  also  needs  sufficient  cues  about  the  flight  path  response  at 
the  main  gear  location  to  succeed  in  this  task.  If  the  initial  and  long-tenn  responses  are  in 
opposite  directions,  it  makes  it  more  difficult  for  the  pilot  to  predict  what  the  flight  path 
response  will  be  for  a  given  input,  and  makes  the  task  of  flaring  the  aircraft  at  the  proper 
time  extremely  challenging.  Aircraft  with  forward  ICR  locations  near  the  pilot  position, 
such  as  the  Concorde  or  Space  Shuttle,  are  known  to  exhibit  poor  flight  path  control 
characteristics  in  the  landing  flare.  This  tendency  produces  a  negative  impact  on  the 
handling  qualities  rating  of  the  aircraft.  This  negative  effect  is  further  enhanced  if  the 
pilot  is  located  far  from  the  center  of  gravity,  because  the  sink  rate  cues  experienced  by 
the  pilot  are  different  from  the  actual  sink  rate  at  the  main  gear  location.  The  ICR 
location  must  therefore  be  carefully  chosen  to  minimize  the  negative  impact  on  the  pilot 
while  maintaining  the  desired  aircraft  capabilities. 

2.5.5  Power  Required  Curve 

The  power  required  curve  is  established  from  the  recognition  that  in  level, 
un-accelerated  flight,  lift  equals  weight  and  thrust  equals  drag.  Power  is  defined  as  the 
rate  of  doing  work,  which  is  a  force  times  a  velocity.  In  this  level,  un-accelerated  flight 
regime,  the  power  must  balance  with  the  drag  force  multiplied  by  the  aircraft  velocity.  If 
the  total  drag  curve  of  an  aircraft  is  multiplied  by  velocity,  then  a  plot  of  power  versus 
airspeed  can  be  formed,  also  known  as  the  “power  required”  curve.  This  curve  is  usually 
defined  from  the  minimum  controllable  airspeed,  or  stall  speed,  to  the  maximum  level 
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flight  speed.  The  total  drag  and  power  required  curves  will  differ,  because  the  former  is  a 
function  of  velocity  squared  and  the  latter  is  a  function  of  velocity  cubed.  This  concept  is 
most  simply  explained  using  the  following  example.  If  velocity  is  doubled,  then  drag 
will  increase  by  four  times,  and  power  required  will  increase  by  an  eight-fold  measure. 
This  is  why  an  increase  in  power  from  an  80%  to  a  100%  setting  does  not  show  a 
corresponding  increase  in  velocity.  The  following  figure  illustrates  an  example  of  a 
power-required  curve. 

Power 


Figure  15  -  Power  Required  Curve 

This  curve  can  be  used  to  clarify  the  concepts  of  the  front  and  back  side  of  the 
power  curve.  The  back  side  of  the  power  curve  is  the  portion  of  the  curve  to  the  left  of 
the  minimum  power  airspeed  (Brandon,  2006).  In  this  region  of  flight,  slower  speeds 
require  more  power,  due  to  the  increased  induced  drag  associated  with  high  angles  of 
attack  at  low  airspeeds.  The  stall  speed  represents  the  slowest  possible  airspeed  for 
controlled  flight.  The  front  side  of  the  power  curve  is  that  portion  of  the  curve  to  the 
right  of  the  minimum  power  airspeed,  and  is  the  flight  region  where  most  aircraft  spend 
the  majority  of  their  time.  The  next  figure  adds  the  power  available  curve  to  power 
required  plot. 
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Power 


Figure  16  -  Power  Available  vs.  Power  Required  Curves 

Thrust  produced  by  a  jet  engine  is  relatively  constant  over  an  aircraft’s  airspeed 
envelope,  and  when  multiplied  by  velocity  represents  the  power  available  to  an  aircraft. 
The  two  power  curves  intersect  at  two  different  points.  The  point  to  the  left  is  the  point 
of  stall.  At  airspeeds  slower  than  this  speed,  the  power  required  exceeds  the  power 
available,  and  the  aircraft  would  be  unable  to  maintain  flight.  The  point  of  intersection  to 
the  right  represents  the  maximum  level  flying  airspeed  of  the  aircraft.  The  aircraft  is 
unable  to  fly  faster  than  this  airspeed  in  level  flight,  because  the  power  required  once 
again  exceeds  the  power  available.  As  aircraft  altitude  increases,  the  power  available 
curve  shifts  down,  until  there  is  only  one  point  where  the  two  curves  intersect.  This 
altitude  is  known  as  the  absolute  ceiling  of  the  aircraft,  and  is  the  highest  altitude  that  the 
aircraft  can  maintain  steady,  level,  un-accelerated  flight. 

As  was  mentioned  earlier,  aircraft  spend  the  majority  of  their  flight  time  on  the 
front  side  of  the  power  curve.  However,  in  certain  flight  conditions,  such  as  powered 
approach  and  landing,  an  aircraft  may  fly  on  the  back  side  of  the  power  curve.  The 
piloting  techniques  associated  with  flying  on  each  side  of  the  power  curve  are  opposite  to 
one  another.  Pilots  flying  the  back  side  technique  use  aircraft  pitch  to  control  airspeed 
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and  power  to  control  flight  path.  If  the  pilot  wants  to  go  faster,  instead  of  increasing  the 
throttle  position,  the  pilot  will  drop  the  nose  of  the  aircraft.  If  the  pilot  wants  to  climb, 
they  will  increase  the  throttle  setting  while  leaving  aircraft  pitch  unchanged.  This 
technique  is  backwards  to  the  normal  flying  convention,  and  the  back  side  of  the  power 
curve  is  therefore  tenned  the  “region  of  reverse  command”.  The  front  side  piloting 
technique  is  the  standard  for  flying.  Pitch  is  used  to  control  flight  path,  and  power 
controls  airspeed.  In  simple  fighter  parlance,  “pull  back  on  the  stick  and  the  houses  get 
smaller,  push  forward  on  the  stick  and  the  houses  get  bigger.” 

Although  most  pilots,  regardless  of  service,  spend  most  of  their  flight  time  on  the 
front  side  of  the  power  curve,  there  are  differences  between  the  services,  particularly  in 
the  realm  of  powered  approach  and  landing.  Air  Force  pilots  will  tend  to  stay  on  the 
front  side  of  the  power  curve,  because  it  represents  a  safer  region  of  flight.  If  a  wind  gust 
slowed  the  aircraft  down  in  this  region,  the  power  required  would  decrease,  and  the  pilot 
would  be  able  to  correct  back  to  a  nonnal  flying  airspeed.  However,  in  the  region  of 
reverse  command,  that  same  wind  gust  would  cause  the  power  required  to  increase,  and  if 
this  new  power  required  exceeded  the  power  available,  the  aircraft  would  not  be  able  to 
maintain  flight.  Due  largely  to  the  requirement  to  land  on  aircraft  carriers,  US  Navy  and 
Marine  Corps  pilots  tend  to  fly  powered  approaches  and  landings  on  the  back  side  of  the 
power  curve.  The  decreased  airspeed  allows  the  aircraft  to  better  land  on  the  ship  and 
catch  the  arresting  cable.  Approach  and  landing  are  demanding  tasks  for  the  pilot,  and 
the  piloting  techniques  used  by  the  respective  service  pilots  when  conducting  these 
operations  tend  to  correspond  to  the  techniques  those  pilots  will  fly  when  highly  tasked  in 
other  flight  conditions.  The  handling  qualities  of  a  new  aircraft  can  be  impacted  by  this 
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preference  for  one  flying  technique  over  another,  and  any  HQ  assessment  should  take  this 
factor  into  account. 

2.6  Pilot  in-the-loop  Simulation  Platforms 

During  the  course  of  this  research  two  different  ground-based  simulation 
platforms  operated  by  AFRL/RBCD  were  used,  the  Infinity  Cube  Simulator  (ICS)  and 
the  Large  Amplitude  Multi -mode  Aerospace  Research  Simulator  (LAMARS). 

2.6.1  Infinity  Cube  Simulator  (ICS) 

The  Infinity  Cube  Simulator  was  a  fixed  based  simulator  with  a  200 
degree  horizontal  and  120  degree  vertical  field  of  view.  Images  were  collimated  to 
between  -0.11  and  0.0083  diopters  to  present  a  focus  distance  close  to  infinity  (Dotter, 
2007).  The  inceptor  was  a  fixed-position  force-sensing  side  stick  that  resembled  an  early 
model  F-16  stick.  The  pilot  would  sit  in  the  seat  and  slide  into  the  simulator.  A  map 
light  was  available  to  provide  needed  illumination  when  making  comments  to  the  test 
cards.  The  test  director  and  control  room  technicians  communicated  with  the  pilot  via  a 
headset.  Simulator  runs  could  be  recorded  with  both  video  and  audio  for  post-test 
analysis,  as  well  as  pilot  inputs  and  aircraft  parameters.  Figure  17  depicts  the  Infinity 
Cube  Simulator  (Dotter,  2007). 
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Figure  17  -  Infinity  Cube  Simulator 

2.6.2  Large  Amplitude  Multi-mode  Aerospace  Research  Simulator 

The  LAMARS  was  briefly  discussed  in  section  2.3,  and  was  depicted  in 
figure  1 1 .  LAMARS  was  a  five  degree-of-freedom  full-motion  simulator.  It  had  a 
simulation  cockpit  enclosed  at  the  end  of  a  thirty-foot  arm  that  could  move  plus  or  minus 
ten  feet  horizontally  or  vertically.  The  simulation  cockpit  could  achieve  up  to  a  3g 
vertical  or  2g  horizontal  acceleration,  and  could  rotate  plus  or  minus  twenty-five  degrees 
in  roll,  pitch,  or  yaw.  The  cockpit  had  both  heads  up  and  heads  down  displays  available 
for  use  during  testing,  and  also  had  the  capability  for  either  a  center  or  side  inceptor 
location.  A  control  room  looked  over  the  simulation  cockpit  and  arm  assembly  and  had 
multiple  displays  depicting  the  aircraft  parameters,  heads  up  and  heads  down  displays, 
and  the  pilot  field  of  view,  which  was  approximately  120  degrees  horizontal  by  40 
degrees  vertical.  The  projectors  in  LAMARS  had  been  modified  to  produce  a  brighter 
image  that  provided  more  realistic  imagery.  There  was  also  a  safety  camera  that  viewed 
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the  pilot  whenever  the  simulator  was  in  motion.  The  test  director  and  control  room 
technicians  communicated  with  the  pilot  via  headset.  Simulator  runs  could  be  recorded 
with  both  video  and  audio  for  post-test  analysis.  Electronic  strip  charts  could  display  and 
record  any  desired  aircraft  parameters. 

2.7  Variable  Stability  Aircraft 

The  use  of  a  variable  stability  aircraft  was  vital  to  this  research,  and  a  brief 
historical  description  follows  to  provide  the  reader  insight  on  the  origins  of  these  aircraft. 
In  1948,  testing  began  to  determine  the  ideal  wing  dihedral  for  the  Ryan  FR-1  Fireball. 
Three  aircraft,  each  with  their  own  wing  dihedral,  were  built  to  detennine  the  best  design 
option.  This  process  was  not  only  labor  and  time  intensive,  but  expensive.  The  desire  for 
a  better  solution  inspired  William  Kauffman  to  develop  the  concept  of  a  variable  stability 
aircraft.  He  postulated  that  the  basic  flight  characteristics  of  an  aircraft  could  be  altered 
by  a  stability  augmentation  system,  so  that  the  handling  qualities  of  several  different 
aircraft,  represented  by  a  broad  range  of  static  and  dynamic  characteristics,  could  be 
simulated  and  tested  in  flight  (Kauffman,  1949).  Eater  that  year,  engineers  at  the  Ames 
Aeronautical  Eaboratory  modified  an  F6F-3  Hellcat  to  become  the  first  variable  stability 
aircraft  ever  constructed. 

The  variable  stability  system  on  this  aircraft  altered  the  effective  wing  dihedral  by 
deflecting  the  ailerons  in  response  to  a  sideslip.  A  modified  control  linkage  allowed  the 
pilot  to  conventionally  control  the  roll  axis  without  feeling  the  variable  stability  system- 
commanded  aileron  deflections.  The  aircraft  was  then  used  in  general  studies  of  lateral- 
directional  flying  qualities  criteria  and  as  an  in-flight  developmental  aircraft  simulator. 
This  second  characteristic  allowed  test  pilots  to  determine  a  new  aircraft’s  handling 
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qualities  before  it  even  flew.  A  prime  example  of  this  ability  was  the  design  of  the  F-104 
Starfighter,  whose  negative  dihedral  wings  were  incorporated  only  after  testing  on  the 
variable  stability  aircraft  (Heinle,  1952). 

In  the  1950’s  high-performance  swept-wing  jet  aircraft  became  the  leading  edge 
of  aviation  technology,  and  caused  an  evolution  in  variable  stability  aircraft,  from  the 
two-axis  variable  F-86  series  of  aircraft  to  the  three-axis  variable  F-100C.  A  variable 
stability  F-86A  and  F-86E  were  used  to  develop  lateral-directional  flying  qualities  for 
these  new  high-perfonnance  aircraft,  while  an  YF-86D  tested  longitudinal  characteristics. 
The  F-100C  became  the  last  high-performance  variable  stability  aircraft  of  the  time 
(Borchers,  1998).  The  next  generation  of  variable  stability  aircraft  then  began  with  the 
NT-33. 

2.7.1  NT-33A 

The  NT-33A  was  a  modified  T-33  trainer  sponsored  by  Wright  Laboratory 
and  used  for  in-flight  simulations.  The  aircraft,  tail  number  0-14120,  was  delivered  to  the 
USAF  in  October  1951  and  transferred  to  the  Calspan  Corporation,  where  it  was 
modified  into  a  variable  stability  aircraft  in  1954.  The  NT-33  A  began  its  first 
engineering  test  flights  in  1959,  after  various  checkouts  and  modifications.  It  possessed 
an  F-94  nose  that  enabled  the  housing  of  the  flight  control  computers  and  recording 
instrumentation.  The  aircraft  trained  hundreds  of  test  pilots  to  evaluate  advanced  aircraft 
and  control  concepts,  analyze  human  factors  concerns,  and  detect  potential  handling 
problems  in  new  aircraft.  Studies  flown  by  the  jet  included  handling  qualities,  pilot- 
vehicle  interaction,  and  flight  control  analyses  of  the  X-15,  X-24,  A-10,  F-15,  F-16,  F-18, 
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F- 1 17,  and  F-22,  among  many  other  American  and  foreign  aircraft  (Brown,  2001).  The 
following  figure  18  is  a  photograph  of  the  NT-33  A  variable  stability  aircraft. 


Figure  18  -  The  NT-33A  Variable  Stability  Aircraft 


The  flight  control  system  on  the  NT-33A  was  a  three  degree-of-freedom, 
response-feedback  system  that  enabled  independent  control  of  the  roll,  pitch,  and  yaw  of 
the  aircraft.  The  flight  control  computer  programmed  the  front  cockpit  flight  controls  to 
perform  according  to  the  simulation  aircraft  flight  characteristics,  so  that  the  pilot  would 
feel  as  if  they  were  flying  different  simulation  aircraft.  A  safety  pilot  in  the  rear  cockpit 
had  standard  controls,  which  allowed  them  to  fly  the  aircraft  in  case  of  a  computer 
malfunction  or  if  the  simulation  aircraft  became  too  demanding  to  control.  The  aircraft 
conducted  its  last  research  in  April  1997,  when  it  retired  with  the  most  flying  hours  of 
any  active  USAF  aircraft.  It  is  now  on  display  at  the  National  Museum  of  the  United 
States  Air  Force. 

2.7.2  NC-131H  TIFS 

The  need  arose  for  another  variable  stability  aircraft  that  would  allow 
testing  of  the  flight  characteristics  of  larger  aircraft.  Calspan,  under  a  Cooperative 
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Research  and  Development  Agreement,  was  tasked  to  develop  the  U.S.  Air  Force  Flight 
Dynamics  Directorate  NC-131H  Total  In-Flight  Simulator  (TIFS)  Aircraft.  This  aircraft 
was  used  to  conduct  the  flight  testing  of  this  research.  Figure  19  shows  the  TIFS  variable 
stability  aircraft  in  flight. 


Figure  19  -  Total  In-Flight  Simulator  in  Flight 

The  TIFS  aircraft  was  developed  in  the  late  1960’s  under  Air  Force  Flight 
Dynamics  Laboratory  sponsorship  in  an  effort  to  help  develop  new  aircraft  and  to 
advance  simulation  technology  for  HQ  research.  The  Federal  Aviation  Administration’s 
(FAA)  interest  in  simulating  Super  Sonic  Transport  (SST)  landing  visibility  also  helped 
initiate  the  project.  Calspan  perfonned  modifications  to  an  Air  Force-furnished  C-131B 
to  convert  it  into  an  in-flight  simulator.  A  separate  simulation  cockpit,  additional  control 
surfaces,  computer-controlled  hydraulic  actuators,  and  turbo-prop  engines  were  all  added. 
The  final  aircraft,  designated  an  NC-131H,  first  flew  in  July  1970.  The  turboprop  engines 
and  propellers  were  replaced  in  1992  and  1994  to  provide  better  performance  and 
maintainability.  The  TIFS  was  a  highly  modified  Convair-580  (USAF  C- 131)  twin 
turboprop  transport,  which  was  used  as  a  six  degree-of-freedom  in-flight  simulator  for 
advanced  flying  qualities  and  display  research.  It  was  also  used  to  demonstrate  advanced 


49 


flight  control  concepts  and  avionics  systems,  and  functioned  as  an  avionics  flying  test  bed 
in  a  separate  configuration. 

According  to  Calspan,  “The  TIFS  unique  features  include  a  separate  two-place 
evaluation  cockpit  and  control  over  all  six  rigid-body  degrees-of-freedom.  Special 
aerodynamic  controls  (including  side-force  and  direct  lift  surfaces)  and  a  model¬ 
following  control  system  permit  the  TIFS  to  produce  motions  at  the  simulation  cockpit 
that  completely  duplicate  the  computed  responses  of  the  simulated  aircraft.  Its  primary 
use  has  been  in  the  development  and  evaluation  of  new  aircraft  flying  qualities,  flight 
controls,  and  cockpit  displays,  as  well  as  general  flight  research  in  these  areas”  (Calspan 
2005).  The  following  figure  20  diagrams  the  capabilities  and  layout  of  the  TIFS. 
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Figure  20  -  TIFS  Capabilities  and  Layout 

The  additional  aerodynamic  controls  of  the  variable  stability  system  (VSS)  on 
TIFS  included  all-moving  side-force  surfaces  on  the  mid  positions  of  the  wings,  and 
direct  lift  flaps,  which  were  outboard  of  the  engine  nacelles.  These  surfaces  worked  in 
combination  with  the  conventional  C- 131  flight  control  surfaces,  the  throttle  servos,  and 
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the  model-following  system  to  provide  full  six  degree-of-freedom  control  (rotational: 
roll,  pitch,  and  yaw;  translational:  normal,  axial,  and  side  forces)  that  completely 
duplicated  the  computed  responses  of  the  simulated  aircraft. 

The  Avionics  Systems  Test  and  Training  Aircraft  (ASTTA)  was  another 
configuration  of  TIFS  with  a  large  avionics  nose  that  was  interchangeable  with  the 
simulation  cockpit  nose  (Peer,  1991).  Developed  in  1985,  the  ASTTA  allowed  the 
addition  of  customer-supplied  large  prototype  radars,  infrared  cameras,  or  other  sensors 
and  equipment.  The  aft  cabin  included  an  instrumented  crew  station  to  accommodate 
system  operators.  In  1998,  extensive  modifications  were  made  to  the  TIFS  simulation 
cockpit  to  accommodate  test  equipment  for  the  eXtemal  Visibility  System  program 
element  of  the  NASA  High  Speed  Research  program  (Babala,  1998)  and  the  synthetic 
vision  component  of  the  Aviation  Safety  program.  TIFS  was  fitted  with  a  new  nose  cap 
and  canopy  to  increase  the  simulation  cockpit  volume  to  accommodate  the  XVS  display 
system  and  a  Collins  X-band  radar  (Calspan,  2005).  The  following  figures  21-24  show 
the  different  TIFS  and  ASTTA  configurations,  as  well  as  a  view  of  the  aft  crew 
compartment. 


Figure  21  -  Front  View  of  Dual-  Figure  22  -  Side  View  of  Lower 

Cockpit  TIFS  Configuration  Cockpit  in  Dual-Cockpit  TIFS 

Configuration 
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Figure  23  -  View  of  Aft  Crew  Figure  24  -  TIFS  with  ASSTA 

Compartment  inside  TIFS  Configuration  Supporting 

Customer  Hardware 

The  TIFS  aircraft  has  been  involved  in  numerous  HQ  assessments  and  research 
and  development  programs  during  its  history.  TIFS  supported  the  Space  Shuttle  Orbiters 
in  several  programs,  and  took  part  in  military  aircraft  development  programs  such  as  the 
B-l,  B-2,  Tacit  Blue,  X-29  and  YF-23.  Calspan  itself  best  describes  the  aircraft’s 
versatility:  “Several  supersonic  transport  aircraft  and  “million-pound”  aircraft 
configuration  programs  for  NASA  and  industry  have  employed  TIFS  for  configuration 
and  control  system  development,  as  well  as  for  visibility  and  sensor  investigations.  TIFS 
has  been  used  for  human  factors  experiments  on  instrumentation;  displays,  control  feel, 
motion  cueing,  and  passenger  ride  sensitivity.  The  ASTTA  configuration  of  TIFS  has 
been  a  training  platfonn  for  test  pilots  and  engineers,  and  has  been  used  for  global 
positioning  system  (GPS),  armament  avionics,  and  remotely  piloted  vehicle  development 
programs.  The  breadth  of  these  programs  illustrates  the  flexibility  of  the  TIFS”  (Calspan, 
2005).  The  aircraft  is  currently  maintained  and  operated  for  the  US  Air  Force  Research 
Laboratory  by  the  Calspan  Flight  Research  Group  in  Niagara  Falls,  New  York.  A 
complete  detailed  description  of  TIFS  is  in  the  TIFS  reference  (Calspan,  2005). 
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2.7.3  NF-16D  (VISTA) 

The  final  variable  stability  aircraft  detailed  in  this  thesis  was  the  Variable- 
Stability  In-Flight  Simulator  Test  Aircraft  (VISTA).  This  thesis  originally  planned  to 
flight  test  on  this  aircraft,  but  it  was  unavailable.  The  NF-16D  was  delivered  to  the 
USAF  in  1995,  and  has  been  operated  by  Calspan,  the  company  who  designed  and 
installed  its  variable  stability  and  other  experimental  systems,  ever  since.  Originally 
based  at  Calspan’s  Flight  Research  Group  in  Niagara  Falls,  New  York,  it  is  currently 
flown  and  maintained  at  Edwards  AFB,  California.  The  USAF  Test  Pilot  School  and 
other  customers  worldwide  use  the  aircraft  as  both  a  research  and  training  tool.  The 
aircraft  provides  many  features,  including:  all-attitude  five  degree-of-freedom  simulation 
capability;  easily  reconfigurable,  fully  instrumented  programmable  controls  and  displays; 
and  an  automatic  safety  monitoring  system.  A  photograph  of  VISTA  during  a  test  flight 
is  shown  in  figure  25  below. 


Figure  25  -  VISTA  in  Flight 

In-flight  simulations  of  prototype  aircraft  are  accomplished  from  the  front  cockpit 
of  the  VISTA.  However,  the  pilot  does  not  require  qualification  in  the  F-16,  because  all 
pilot-in-command  displays  and  controls  are  relocated  to  the  aft  cockpit,  where  the  safety 
pilot  monitors  the  flight.  This  safety  pilot,  backed-up  by  a  quad-redundant  automatic 
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VISTA  Integrity  Monitor,  ensures  that  tests  do  not  exceed  the  limitations  of  the 
simulation  system  or  the  aircraft  itself.  The  following  figure  26  diagrams  the  capabilities 
and  layout  of  the  VISTA. 
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Figure  26  -  Capabilities  of  the  Variable-Stability  In-Flight  Simulator  Test  Aircraft 
The  VISTA  has  a  programmable  simulation  system  that  allows  for  efficient 

checkout  of  different  software  loads.  Changes  to  the  system  do  not  require  extensive 
verification  and  validation  testing,  because  the  simulation  system  is  not  critical  to  safety- 
of-fhght.  A  suite  of  digital  computers  connected  by  dedicated  1553  data  buses  provides 
the  “heart  of  the  simulation  system”  (Calspan- VISTA,  2006).  Aircraft  parameters  needed 
for  testing  are  digitally  recorded  and  can  be  transmitted  in  real-time  via  a  telemetry 
downlink.  VISTA  can  also  integrate  weapons  systems  and  tactical  display  concepts  into 
the  simulations  via  wing  hard  points  and  APG-68  targeting  radar.  This  aircraft  has  been  a 
fundamental  part  of  the  developmental  testing  of  cutting-edge  aircraft,  including  the  F-35 
Joint  Strike  Fighter,  the  Indian  Light  Combat  Aircraft,  and  the  X-38.  This  variable 
stability  aircraft  represents  a  significant  asset  to  the  research  and  development  of  new 
fighter-type  aircraft  and  their  corresponding  weapons  systems. 
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2.8  Summary 

This  chapter  sought  to  explain  the  background  infonnation  that  required 
understanding  in  order  to  fully  comprehend  this  study.  It  began  with  a  synopsis  of 
handling  qualities  scales,  including  the  Cooper-Harper  rating  scale,  and  detailed  the  use 
of  each.  It  then  detailed  the  impetus  for  and  research  behind  the  next  generation  of  long- 
range  strike  aircraft,  focusing  on  the  2006  QDR  and  NGC’s  STAV  design.  The  related 
research  of  the  cranked-arrow  delta  wing  F-16XL  program  and  the  Space  Shuttle  Orbiter/ 
TIFS  program  were  discussed.  Issues  that  impacted  handling  qualities  during  powered 
approach  were  then  covered.  This  included  longitudinal  and  lateral/  directional  aircraft 
stability  from  both  a  static  and  dynamic  viewpoint  and  the  concepts  of  dynamic  inversion 
in  flight  controls,  center  of  rotation,  and  the  front  and  back  side  of  the  power  required 
curve.  The  two  ground-based  pilot  in-the-loop  simulation  platforms  were  then  detailed. 
The  chapter  concluded  with  a  historical  review  of  variable  stability  aircraft,  including 
detailed  information  on  three  of  the  most  important:  the  NT-33A,  the  NC-131H  TIFS, 
and  the  NF-16D  VISTA. 
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3.0  Testing  Methods  and  Procedures 

3.1  Overview 

This  chapter  outlines  the  different  methods  and  procedures  conducted  throughout 
this  study’s  test  research.  The  discussion  first  focuses  on  the  scope  and  assumptions  of 
this  thesis.  It  then  covers  the  overall  general  test  methodology,  including  the  initial  test 
procedures  developed  during  initial  Large  Amplitude  Multi-mode  Aerospace  Research 
Simulator  (LAMARS)  testing  by  the  Northrop  Grumman  Corporation  (NGC).  It  also 
describes  the  specific  integration  of  the  different  Supersonic  Tailless  Air  Vehicle  (STAV) 
models  and  the  test  specific  procedures  for  each  portion  of  the  testing:  Infinity  Cube 
simulator  (ICS),  LAMARS,  and  Total  In-Flight  Simulator  (TIFS)  flight  tests.  These  test 
specific  procedures  included  both  the  tasks  and  test  cards  that  each  pilot  flew  as  well  as 
the  desired  parameters  and  constraints  used  in  each  test  section.  Finally,  the  data  analysis 
plans  for  the  test  results  of  each  section  are  all  explained  in  detail. 

3.2  Scope/  Assumptions 

The  main  factor  in  the  formation  and  conduct  of  this  thesis  was  the  requirement 
for  actual  flight-testing  of  the  thesis  topic.  This  flight-testing  would  have  to  be  conducted 
in  accordance  with  the  guidance  set  forth  by  the  USAF  Test  Pilot  School  (TPS)  test 
management  project  (TMP)  program.  The  thesis  topic  would  have  to  fulfill  the 
requirements  of  both  an  AFIT  thesis  and  a  TPS  TMP.  The  TMP  to  evaluate  the  STAV 
handling  qualities  was  named  project  HAVE  STAV.  While  a  handling  qualities 
evaluation  of  the  entire  STAV  flight  envelope  would  have  been  desirable,  a  program  of 
such  magnitude  would  have  far  exceeded  the  scope  of  this  thesis  and  the  flight  test 
capabilities  of  a  single  TMP.  Rather  than  provide  some  general  qualitative  assessment  of 
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STAV  handling  qualities,  this  thesis  focused  on  a  handling  qualities  (HQ)  evaluation  of  a 
specific  low-speed  region  of  flight,  that  of  powered  approach  and  landing.  In  order  to  fly 
this  thesis,  a  variable  stability  aircraft  capable  of  simulating  the  STAV  flight  dynamics 
and  of  conducting  the  desired  flight  tests  had  to  be  selected.  Originally,  this  study 
planned  on  using  the  Variable-Stability  In-Flight  Simulator  Test  Aircraft  (VISTA)  at 
TPS,  but  this  was  not  possible  due  to  aircraft  availability.  The  selection  of  the  Total  In- 
Flight  Simulator  (TIFS)  as  the  test  aircraft  was  subject  to  and  met  all  of  the  cost, 
availability,  and  safety  concerns  involved  in  this  flight-test  program.  The  decision  to  use 
this  aircraft  helped  to  refine  the  desired  test  objectives  to  the  ones  used  in  the  conduct  of 
this  thesis.  Several  other  limiting  assumptions  were  made  to  maintain  both  the  scope  and 
focus  of  the  HQ  evaluation  of  the  STAV  in  this  thesis. 

All  HQ  evaluations,  both  qualitative  and  quantitative,  were  based  off  of  the 
Cooper-Harper  rating  (CHR)  scale,  the  primary  rating  scale  used  by  modern  USAF  test 
pilots.  Even  though  Cooper-Harper  ratings  are  not  normally  averaged,  for  the  purposes 
of  the  ICS  testing  it  was  assumed  that  the  CHR  could  be  averaged  in  order  to  statistically 
compare  different  pilot  groups.  For  this  research,  the  rating  on  the  pilot  in-the-loop 
oscillation  (PIO)  scale  was  assumed  to  be  1  unless  a  PIO  was  encountered  or  if  a  PIO- 
tendency  was  specifically  noted  by  the  pilot.  There  was  no  thrust  vectoring  used  in  the 
STAV  model,  and  an  initial  30%  spoiler  bias  setting  was  used  in  all  approach  and  landing 
tests.  This  value  was  selected  because  it  provided  the  best  speed  stability  on  powered 
approach  and  landing  during  the  full-motion  simulations  conducted  by  the  NGC  on 
LAMARS.  The  ICS  was  selected  over  LAMARS  for  the  initial  testing  in  this  study 
because  it  provided  better  visual  cues  and  capability  in  the  powered  approach  and  landing 
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arena,  and  was  easier  to  use  with  a  large  group  of  pilots.  These  visual  cues  were  assumed 
to  have  more  of  an  impact  on  pilot  opinion  than  the  subtle  motion  cues  experienced  in 
LAMARS,  especially  on  landing.  All  cockpit  vision  issues  associated  with  angle  of 
attack  or  cockpit  location  were  not  included  in  this  testing.  The  side  stick  used  in  the 
Infinity  Cube  Simulator  was  considered  to  have  minimal  impact  on  the  HQ  evaluations 
conducted  prior  to  TPS. 

The  testing  throughout  this  study,  including  both  simulation  and  flight  testing, 
involved  two  different  NGC  long-range  strike  concept  STAV  models  that  used  dynamic 
inversion  as  part  of  the  flight  control  algorithm.  The  STAV  model  used  in  initial 
LAMARS  and  ICS  testing  did  not  include  ground  effect  or  gear  modeling,  which  made 
the  actual  landing  HQ  of  the  STAV  impossible  to  specifically  determine.  However,  this 
evaluation  was  used  to  generate  an  approximate  HQ  rating  for  both  approach  and  landing. 
The  Version  2  STAV  model,  which  from  now  on  will  be  referred  to  as  the  baseline 
STAV  model,  included  both  ground  effect  and  gear  modeling  and  was  tested  in  both 
LAMARS  and  TIFS.  Although  two  different  models  were  tested  during  the  course  of 
this  thesis,  it  was  assumed  that  results  from  testing  the  first  model  could  be  compared  to 
results  from  the  second  model,  particularly  from  a  qualitative  sense,  and  that  lessons 
learned  from  initial  testing  could  be  applied  to  subsequent  testing.  For  the  purposes  of 
this  study,  dynamic  inversion  was  assumed  to  be  a  viable  flight  control  option,  and  the 
structural  issues  associated  with  control  surface  movements  were  considered  to  have  a 
negligible  impact  on  the  flight  control  system. 

The  pilots  used  throughout  the  test  program  were  not  all  test  pilots.  The  pilots 
available  at  the  Air  Force  Institute  of  Technology  (AFIT)  encompassed  a  broad  range: 
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from  heavy  to  fighter,  from  zero  to  significant  test  experience,  and  from  civilian  to  three 
different  services.  These  varying  backgrounds  seemed  to  preclude  repeatability  in  test 
data,  so  a  technique  was  used  to  set  a  baseline  for  the  ICS  testing.  General  HQ 
information  and  CHR  procedures  were  briefed  to  all  pilots  before  testing.  In  addition  to 
conducting  an  HQ  evaluation  of  the  STAV,  these  pilots  also  flew  a  T-38  model  flying  the 
same  maneuvers.  The  T-38  model  was  a  hi-fidelity  model  that  included  ground  effect 
and  had  been  tested  to  ensure  it  closely  resembled  actual  T-38  flight  characteristics.  The 
pilots  rated  both  aircraft,  and  their  HQ  evaluations  were  compared  to  historical 
information  about  the  T-38  to  establish  a  baseline  for  the  non-test-pilot  raters.  It  was 
assumed  that  this  would  allow  a  HQ  evaluation  by  non-test-pilots  to  be  comparable  to  an 
HQ  evaluation  conducted  strictly  by  test  pilots. 

After  beginning  TPS,  it  became  evident  that  the  number  of  pilots  able  to 
participate  in  the  HAVE  STAV  TMP  would  be  limited  to  three,  much  less  than  the  large 
pilot  pool  tested  at  AFIT.  Also,  the  STAV  model  itself  could  not  be  altered  due  to 
proprietary  reasons.  However,  the  inputs  going  into  the  model  could  be  altered,  and  the 
test  objectives  changed  from  testing  different  types  of  pilots  to  testing  different  control  or 
feel  systems.  This  minor  migration  in  test  objectives  was  assumed  to  enhance  the  overall 
scope  and  quality  of  the  research. 

Finally,  the  initial  flight  test  matrix  called  for  at  least  ten  test  flights  to  conduct  the 
HQ  evaluation  on  TIFS,  but  due  to  monetary  and  time  constraints  this  was  reduced  to  ten 
hours  of  flight  time.  These  constraints  also  prevented  the  implementation  of  a  Heads  Up 
Display  (HUD)  in  the  TIFS  cockpit.  The  decision  was  made  to  forgo  a  HUD  in 
LAMARS  as  well,  even  though  it  had  the  capability  to  use  one,  so  that  the  cockpit  layout 
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in  LAMARS  would  match  that  of  TIFS.  Testing  in  LAMARS  and  TIFS  instead  used  a 
heads  down  display,  accompanied  by  altitude  calls  from  a  test  engineer.  The  flexibility  in 
the  test  program  allowed  the  TMP  team  to  meet  all  of  the  flight  test  objectives  with  the 
limited  flight  test  time. 

3.3  General  Test  Methodology 

In  the  spring  of  2006,  the  Northrop  Grumman  Corporation  (NGC)  conducted 
LAMARS  testing  on  a  concept  STAV,  in  an  effort  to  evaluate  the  handling  qualities  at 
various  high  and  low  speed  flight  regimes.  In  order  to  accomplish  this  evaluation,  three 
different  test  pilots  and  two  other  USAF  pilots  (including  the  author)  were  given  a  set  of 
tasks  to  perfonn.  These  tasks  were  designed  to  be  operationally  valid  maneuvers  that  a 
new  strike  aircraft  would  be  expected  to  accomplish  on  a  given  mission.  Each  task  had 
several  performance  metrics  that  measured  the  pilot’s  ability  to  successfully  complete  the 
maneuver.  At  the  completion  of  each  task,  the  pilot  rated  the  handling  qualities  of  the 
aircraft  according  to  both  the  performance  achieved  and  the  workload  required  to 
accomplish  the  task.  The  tasks  were  then  repeated  at  different  airspeeds  and  altitudes  in 
an  effort  to  more  completely  explore  the  aircraft  mission  envelope.  This  initial  NGC 
LAMARS  testing  served  as  the  basis  for  the  research  conducted  in  this  thesis. 

The  results  from  this  initial  NGC  testing  revealed  that  the  control  laws  and 
aerodynamic  effectiveness  of  the  control  surfaces  were  stressed  the  most  during  the  low- 
speed  approach  and  landing  test  conditions.  It  also  showed  that  piloting  technique 
seemed  to  play  a  large  role  in  the  perceived  HQ.  In  response  to  these  findings,  a  research 
effort  was  initiated  by  the  author  to  continue  low-speed  approach  and  landing  testing. 
The  testing  followed  the  same  format  as  the  initial  NGC  LAMARS  testing,  where  a  pilot 
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conducted  a  series  of  tasks  and  then  rated  the  HQ  based  on  the  workload  and 
performance  achieved.  The  different  STAV  models  provided  by  the  NGC  to  AFIT  and 
Calspan  allowed  the  construction  of  several  test  profiles.  These  profiles  were  then  tested 
in  one  of  three  platforms:  the  Infinity  Cube  Simulator  at  the  Air  Force  Research 
Laboratory  (AFRL)  using  a  single  throttle  and  side  inceptor  with  the  initial  STAV  model; 
the  LAMARS  full-motion  simulator  at  AFRL  using  two  throttles  and  a  center  inceptor 
with  the  updated  (baseline)  STAV  model;  and  the  NC-131H  TIFS  variable  stability 
aircraft  using  two  throttles  and  a  center  inceptor  with  the  updated  (baseline)  STAV 
model. 

The  Infinity  Cube  Simulator  testing  focused  on  how  different  pilot  backgrounds 
(i.e.  time,  type,  service)  influenced  perfonnance  and  HQ  evaluations.  In  addition  to  the 
initial  STAV  model,  a  T-38  model  was  also  tested,  in  order  to  set  a  baseline  for  all  the 
pilots.  This  allowed  the  data  to  be  reduced  at  both  an  overall  ratings  level  and  according 
to  each  pilot  characteristic.  This  was  done  to  reveal  any  trends  or  tendencies  for  certain 
pilots  to  rate  similar  tests  differently. 

As  mentioned  in  the  previous  section,  the  focus  of  testing  changed  after  the  ICS  to 
studying  the  impact  of  different  control  systems  on  HQ.  This  was  due  to  the  limited 
number  of  pilots  on  the  HAVE  STAV  TMP  team.  Prior  to  testing  in  LAMARS,  each 
HAVE  STAV  pilot  flew  the  different  test  tasks  in  a  TPS  T-38  so  that  they  could  become 
familiar  with  them  and  validate  that  they  were  operationally  valid  and  safe.  The 
LAMARS  was  selected  over  the  ICS  by  the  TMP  team  so  that  a  direct  comparison  of  test 
data  could  be  made  with  a  second  round  of  NGC  LAMARS  testing.  Since  the  LAMARS 
testing  was  conducted  in  preparation  for  flight  testing,  the  TMP  team  wanted  a  higher 
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fidelity  simulation  with  motion,  so  that  these  motions  could  be  compared  to  those 
experienced  on  TIFS.  The  LAMARS  testing  focused  on  familiarizing  the  HAVE  STAV 
pilots  with  the  baseline  STAV  model  and  on  developing  an  alternate  control  or  feel 
system  that  could  be  flight  tested  on  TIFS  and  compared  to  the  baseline  STAV  model. 
First,  the  flying  qualities  of  the  baseline  STAV  model  as  implemented  on  LAMARS  were 
compared  to  those  found  in  the  second  round  of  NGC  LAMARS  testing  to  ensure  that  the 
results  closely  matched.  Then,  the  specific  type  of  feedback  control  for  the  baseline 
STAV  model  was  chosen,  after  which  an  alternate  control  system  was  optimized.  This 
LAMARS  optimized  model  was  then  compared  to  the  baseline  STAV  model. 

The  TIFS  flight  testing  also  compared  the  flying  qualities  of  the  TIFS- 
implemented  STAV  model  to  the  second  round  of  NGC  LAMARS  testing,  again  to 
ensure  that  the  model  following  was  accurate  to  the  predicted  STAV  response.  Both  the 
baseline  and  LAMARS  optimized  models  were  then  flown  and  compared  to  see  if  the  HQ 
were  better  with  one  model  than  the  other.  Once  the  data  were  reduced,  observational 
and  interpretive  analysis  was  conducted  to  best  summarize  the  test  results  for  each  test 
section. 

3.4  Infinity  Cube  Simulator  (ICS)  Testing 

In  October  of  2006,  a  series  of  simulator  tests  was  conducted  in  the  ICS  by  a 
group  of  nineteen  pilots  of  varying  flying  backgrounds.  The  testing  was  divided  into  two 
phases,  a  preparation  and  an  execution  phase.  During  the  preparation  phase,  the  test  tasks 
were  defined  and  the  overall  test  plan  was  developed.  The  test  plan  included  a  straight-in 
precision  approach  and  landing  task,  a  lateral  offset  landing  task,  and  a  vertical  offset 
landing  task.  All  tasks  were  designed  to  land  the  aircraft  1 ,000  feet  down  the  runway  on 
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centerline.  During  the  precision  approach  task,  the  pilot  had  to  maintain  approach 
airspeed  while  flying  down  a  specified  glideslope.  During  each  of  the  landing  tasks,  the 
pilots  had  to  land  within  a  designated  zone  on  the  runway  while  meeting  different  sink 
rate,  airspeed,  heading,  and  bank  angle  requirements.  During  the  offset  landing  tasks,  the 
pilots  would  maintain  a  course  or  glideslope  that  would  result  in  either  a  lateral  or 
vertical  offset  from  the  runway.  At  300  feet  above  ground  level  (AGL),  the  pilot  would 
either  correct  laterally  back  to  runway  centerline  or  would  vertically  correct  to  land  the 
proper  distance  down  the  runway.  These  tasks  were  some  of  the  same  as  those  conducted 
during  the  initial  NGC  LAMARS  testing,  and  also  had  almost  identical  performance 
criteria,  which  were  set  based  off  of  previous  research  conducted  during  the  high-speed 
civil  transport  program.  The  only  perfonnance  criterion  that  differed  was  the  touchdown 
sink  rate,  which  was  changed  to  higher  values  after  looking  at  actual  performance 
achieved  during  the  initial  NGC  LAMARS  testing.  The  tasks  were  designed  to  mimic 
the  operational  conditions  of  flying  a  precision  instrument  approach,  a  non-precision 
approach  that  brings  the  aircraft  in  offset  with  the  runway  laterally,  and  an  approach 
where  the  aircraft  breaks  out  of  the  weather  at  a  higher  than  normal  glideslope.  As  the 
tasks  increased  in  difficulty,  the  pilot  gain  increased  in  an  effort  to  reveal  any  poor  HQ 
not  evident  in  lower  gain  tasks.  All  tests  would  be  flown  with  a  30%  spoiler  bias, 
meaning  that  the  spoilers  would  be  extended  30%  at  all  times  during  the  approach  and 
landing.  Previous  testing  showed  this  provided  better  speed  stability  and  control  during 
approach  and  landing. 

After  defining  the  tasks,  a  test  plan  was  created  that  defined  both  test  conduct  and 
test  goals.  The  test  goals  were  to  establish  if  piloting  technique  or  background  influenced 
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how  the  HQ  were  rated  and  to  provide  an  overall  test  methodology  to  be  used  during  the 
TPS  curriculum.  In  order  to  detennine  the  role  piloting  technique  played  on  the 
perceived  aircraft  HQ,  the  approach  and  landing  tasks  were  conducted  at  two  different 
approach  airspeeds,  175  and  195  knots.  While  these  airspeeds  were  actually  both  on  the 
back  side  of  the  power  curve,  they  were  set  far  enough  apart  to  simulate  both  front  and 
back  side  of  the  power  curve  conditions.  The  offsets  used  in  the  initial  NGC  LAMARS 
testing  were  used  again,  as  they  represented  operationally  valid  maneuvers.  The  pilot 
pool  at  the  Air  Force  Institute  of  Technology  (AFIT)  varied  widely  in  background  and 
experience.  In  order  to  establish  some  sort  of  baseline  for  all  of  the  pilots,  an  aircraft 
with  known  handling  qualities  was  used,  the  T-38.  All  of  the  tasks  were  accomplished 
flying  both  the  STAV  model  and  the  T-38.  After  defining  the  tasks  and  scope  of  the  test, 
a  series  of  test  cards  was  created.  They  each  included  the  performance  criteria  for  the 
task,  the  directions  for  accomplishing  the  task,  and  areas  for  pilot  comments  and  Cooper- 
Harper  rating.  The  ICS  test  cards  are  located  in  appendix  C,  figures  C-l  through  C-3. 

The  simulator  used  in  testing,  the  ICS,  was  a  fixed-base  simulator  that  provided 
outstanding  visuals  over  a  200  degree  field  of  view.  The  ICS  was  selected  over 
LAMARS  for  the  initial  testing  in  this  study  because  it  provided  better  visual  cues  and 
capability  in  the  powered  approach  and  landing  arena,  and  was  easier  to  use  with  a  large 
group  of  pilots.  These  visual  cues  were  assumed  to  have  more  of  an  impact  on  pilot 
opinion  than  the  subtle  motion  cues  experienced  in  LAMARS,  especially  on  landing. 
Prior  to  the  test  execution,  all  of  the  pilots  involved  in  testing  were  briefed  in  detail  on  the 
tasks,  the  performance  criteria,  and  the  simulator  operation.  Each  pilot  also  received 
instruction  on  the  CHR  scale  and  how  to  use  it.  This  instruction  conformed  to  the 
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curriculum  at  the  USAF  Test  Pilot  School.  The  pilots  were  briefed  that  they  would  be 
flying  two  different  flight  control  models;  they  were  not  told  that  one  of  the  models  was 
the  T-38.  That  infonnation  was  purposely  withheld  in  order  to  maintain  an  unbiased 
opinion  prior  to  testing.  Each  pilot  was  instructed  to  study  the  test  tasks  and  the  CHR 
scale  prior  to  testing.  In  order  to  ensure  the  proper  motivation  levels,  the  pilots  were 
briefed  that  the  best  and  worst  performers  would  be  highlighted  and  revealed,  a  fact  that 
produced  nineteen  well-prepared  pilots. 

In  the  week  prior  to  testing,  the  T-38  and  STAV  models  were  loaded  onto  the  ICS 
and  calibrated.  Due  to  modeling  constraints  and  availability,  an  F-16  HUD  was  used 
with  the  T-38  model  and  a  C-17  HUD  was  used  with  the  STAV  model.  During  the 
execution  phase  of  testing,  each  pilot  was  in  the  simulator  for  approximately  forty-five 
minutes.  Whenever  a  new  model  was  introduced,  the  pilots  flew  a  practice  approach  and 
landing  before  conducting  any  approaches  for  data.  The  pilots  were  briefed  on  the  HUD 
differences  between  the  two  models.  The  T-38  model  was  always  flown  first,  and  after 
the  practice  approach  each  pilot  flew  the  precision  approach  and  landing,  lateral  offset 
landing,  and  vertical  offset  landing  tasks.  The  testing  was  conducted  over  a  three  day 
period,  and  the  pilots  were  divided  evenly  each  day  according  to  their  background.  After 
flying  the  T-38  model  at  175  knots,  the  STAV  model  was  flown  at  175  and  195  knots 
approach  speed.  Half  of  each  pilot  group  flew  the  175  knot  approaches  prior  to  the  195 
knot  approaches.  The  other  half  of  each  pilot  group  flew  in  the  reverse  order.  This  was 
done  to  counter  any  overall  handling  qualities  improvement  brought  on  solely  by 
learning.  The  pilot  comments  and  aircraft  parameters  were  recorded  for  each  test  run  on 
a  computer  file,  an  audio  file,  and  a  video  file.  For  the  approach  task  performance 
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criteria,  the  aircraft  parameters  when  passing  through  1000  feet  AGL  were  recorded  and 
used  to  ascertain  pilot  perfonnance.  For  the  landing  tasks,  the  aircraft  parameters  at 
touchdown  were  used  to  measure  pilot  perfonnance.  After  the  pilots  made  comments  and 
saw  the  performance  achieved,  they  gave  two  CHR  for  each  run,  one  for  the  longitudinal 
axis  and  one  for  the  lateral  axis.  The  two  CHR  were  given  to  highlight  any  hidden  HQ 
deficiencies  that  occurred  in  a  specific  axis.  After  completing  all  of  the  test  tasks,  each 
pilot  was  briefed  to  not  discuss  the  testing  with  any  other  pilots  until  after  all  ICS  testing 
was  complete. 

The  data  collected  during  the  ICS  testing  was  reduced  and  analyzed  using  a  data 
analysis  plan  created  prior  to  test  execution.  During  testing,  each  data  run  was  given  a 
number,  so  that  it  could  be  more  easily  organized  after  test  completion.  On  each  data 
run,  a  hard  copy  of  a  test  card  was  used  by  the  test  conductor  to  record  pilot  comments, 
initial  performance  parameters,  and  CHR.  During  testing,  runs  that  were  noted  by  the 
test  conductor  as  particularly  interesting  were  noted,  so  that  they  could  be  pulled  from  all 
the  other  runs  after  testing.  The  audio  comments  and  performance  achieved  on  each  run 
were  reviewed  to  make  sure  that  the  final  CHR  was  proper.  The  computer  files  were 
recorded  in  a  manner  that  they  could  be  easily  transferred  to  an  Excel  spreadsheet  for 
data  reduction.  The  data  were  divided  first  by  aircraft,  and  then  by  each  pilot  group.  The 
overall  CHR  and  perfonnance  achieved  in  each  aircraft  was  recorded  for  each  task.  The 
data  were  then  broken  down  by  pilot  type,  experience,  and  service.  These  three  pilot 
classifications  each  had  two  groups:  fighter  and  heavy  for  type,  test  and  non- test  for 
experience,  and  Air  Force  and  Navy/  Marine  Corps/  Civilian  for  service.  Both  Navy  and 
Marine  Corps  pilots  were  considered  to  be  Navy  pilots,  and  the  civilian  pilot  was 
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considered  to  be  a  heavy  pilot.  The  data  were  analyzed  to  see  if  any  pilot  group  rated  the 
HQ  vastly  different  from  another  group,  or  if  they  preferred  a  certain  approach  airspeed 
over  another.  It  was  also  analyzed  to  see  if  one  pilot  group  was  able  to  fly  with  greater 
precision  than  another,  and  have  better  perfonnance  parameters.  Each  performance 
criterion  was  weighted  equally  and  the  pilot  groups  were  compared  using  a  term  called 
parameter  accuracy.  The  pilot  groups  were  analyzed  to  see  if  one  group  learned  faster 
than  another  (i.e.  the  STAV  model  CHR  improved)  as  testing  progressed.  HQ  results 
from  initial  LAMARS  testing  were  compared  to  ICS  testing  HQ  results.  The 
aerodynamic  characteristics  of  the  STAV  model,  including  factors  like  short  period 
damping  and  phugoid  time  to  double,  were  used  to  determine  the  predicted  HQ  of  the 
STAV,  which  were  then  compared  to  the  HQ  found  in  testing.  This  analysis  was  then 
used  to  make  conclusions  and  recommendations  for  the  ICS  testing. 

3.5  LAMARS  Simulator  Testing 

Testing  of  the  STAV  model  was  conducted  by  the  HAVE  STAV  TMP  team  in  the 
LAMARS  full  motion  simulator  on  6-7  August  2007.  As  mentioned  previously, 
LAMARS  was  selected  over  the  ICS  so  that  a  direct  comparison  of  test  data  could  be 
made  with  the  second  round  of  NGC  LAMARS  testing  conducted  in  November  2006. 
Since  the  LAMARS  testing  was  conducted  in  preparation  for  flight  testing,  the  TMP  team 
wanted  a  higher  fidelity  simulation  with  motion,  so  that  these  motions  could  be  compared 
to  those  experienced  on  TIPS.  The  main  objective  was  to  identify  an  optimized  flight 
control  system,  feel  system,  or  technique  to  flight  test  on  the  TIPS  in  addition  to  the 
baseline  STAV  model.  This  simulator  testing  was  used  to  familiarize  the  test  team  with 
the  STAV  model  and  test  tasks  prior  to  flight  testing  on  TIPS.  In  order  to  better  replicate 
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the  capabilities  of  the  TIFS  cockpit,  a  center  inceptor  location  was  chosen  and  a  HUD 
was  not  employed.  The  test  cards  used  in  LAMARS  testing  can  be  found  in  appendix  C, 
figures  C-4  through  C-9.  The  tasks  were  the  same  as  those  used  in  the  ICS  except  for  the 
normal  approach  and  landing  task.  This  task  became  just  a  single  evaluation,  instead  of 
an  approach  evaluation  and  a  landing  evaluation.  Also,  a  sole  CHR  was  assigned  to  each 
task,  instead  of  a  lateral  and  longitudinal  CHR.  All  tasks  were  again  designed  to  land  the 
aircraft  1,000  feet  down  the  runway  on  centerline.  The  performance  criteria  were  also  the 
same  except  for  the  sink  rate  criteria,  which  were  decreased  to  account  for  the  STAV 
landing  gear  structural  capabilities.  The  STAV  model  tests  all  began  with  a  30%  spoiler 
bias,  for  the  same  reasons  mentioned  previously. 

A  factorial  design  method  (Montgomery,  2005)  was  initially  used  with  four 
variables  (pilot,  offset,  crosswind,  and  approach  airspeed)  to  find  the  optimal  test  matrix 
where  the  most  significant  variable  interactions  would  be  identified.  This  matrix  was 
executed  on  LAMARS  by  the  TMP  team  to  verily  predictions  and  to  narrow  down  the 
actual  test  matrix  for  flight  testing.  LAMARS  testing  was  conducted  by  the  TMP  team  in 
three  phases.  The  first  phase  focused  on  an  investigation  of  the  flying  qualities  of  the 
baseline  STAV  model  and  a  comparison  of  the  alpha,  gamma,  and  q-command  control 
systems.  The  test  team  used  a  series  of  impulses,  steps,  and  semi-closed-loop  capture 
tasks  in  each  axis  to  detennine  the  flying  qualities  of  the  baseline  STAV  model  as 
implemented  on  LAMARS,  and  compared  the  results  to  those  found  in  the  second  round 
of  NGC  LAMARS  testing  to  ensure  that  the  results  closely  matched.  The  second  round 
of  NGC  LAMARS  testing  also  investigated  the  angle  of  attack  (alpha-command),  flight- 
path  angle  (gamma-command),  and  a  pitch-rate  (q-command)  control  systems.  It 
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indicated  that  the  optimal  flying  qualities  during  powered  approach  and  landing  tasks 
were  obtained  using  an  angle  of  attack  (alpha-command)  control  system.  Each  HAVE 
STAV  pilot  conducted  a  limited  evaluation  of  the  baseline  STAV  model  with  each  of 
these  control  systems  to  determine  which  the  best  was  and  which  warranted  further 
investigation  on  TIFS.  The  pilots  flew  two  or  three  practice  approaches  before  flying  the 
tasks  for  data.  This  procedure  was  done  to  familiarize  the  pilot  with  the  sight-picture  of 
the  flare  and  pacing  of  the  approach  and  landing.  Each  pilot  developed  a  technique  for 
accomplishing  the  flare  during  this  first  phase,  after  which  the  pilots  decided  on  a 
standardized  flare  technique.  Each  pilot  accomplished  the  precision  approach  and  lateral 
offset  tasks  with  and  without  crosswind,  as  well  as  a  vertical  offset  landing  task.  These 
maneuvers  were  accomplished  to  see  if  offsets  in  different  axes  produced  different 
workloads  for  the  pilots.  These  simulations  were  accomplished  using  only  a  heads  down 
display,  because  TIFS  did  not  have  a  heads  up  display  (HUD). 

The  initial  and  second  rounds  of  NGC  LAMARS  testing  revealed  the  powered 
approach  and  landing  tasks  that  involved  a  lateral  offset  or  high  crosswinds  demonstrated 
a  high  pilot  workload  and  potential  for  pilot  in-the-loop  oscillation  (PIO).  The  forward 
location  of  the  instantaneous  center  of  rotation  and  the  associated  flight  path  response 
was  the  likely  reason  for  this  PIO  potential.  As  the  pilot  tried  to  make  aggressive 
corrections  back  to  the  runway,  the  initial  motion  was  in  the  opposite  direction  of  the 
commanded  motion  in  both  pitch  and  yaw.  In  an  effort  to  improve  aircraft  handling 
qualities,  the  effects  of  increasing  longitudinal  inceptor  force  gradients  and  the  effects  of 
spoiler  retraction  on  flare  characteristics  were  studied  by  the  TMP  team  in  phase  two  of 
LAMARS  testing.  An  increased  force  gradient  would  reduce  the  tendency  to  over- 
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control,  and  the  spoiler  retraction  would  counter  some  of  the  moment  generated  when 
pulling  aft  on  the  inceptor,  potentially  shifting  the  instantaneous  center  of  rotation  and 
improving  handling  qualities  during  the  flare. 

This  second  phase  involved  modifying  the  feedback  control  system  judged  best 
during  phase  one  of  the  testing.  This  modification  involved  automatically  increasing  the 
force  gradient  in  the  longitudinal  axis  when  passing  through  a  set  AGL  altitude.  Both  the 
value  of  the  force  gradient  and  the  altitude  of  the  gradient  change  were  varied  in  order  to 
yield  a  more  repeatable  and  predictable  flare.  The  first  pilot  to  test  the  system  conducted 
the  test  tasks  while  varying  both  the  altitude  and  value  of  the  force  gradient  change.  The 
values  judged  best  by  the  first  pilot  were  passed  on  to  the  next  pilot,  who  began  with 
these  values  and  altered  them  before  passing  them  on  to  the  next  pilot.  This  process 
continued  until  the  values  were  set  to  an  optimized  level.  To  detennine  the  effects  of 
spoiler  retraction,  the  force  gradient  was  reset  to  the  baseline  and  the  spoilers  were 
automatically  retracted  when  passing  through  a  certain  AGL  altitude.  The  altitude  of  this 
retraction  was  optimized  in  the  same  manner  as  the  force  gradient  changes,  in  an  effort  to 
achieve  complete  spoiler  retraction  as  touchdown  occurred.  The  two  modifications  were 
then  made  simultaneously,  and  the  pilots  again  assigned  a  CHR  according  to  workload 
and  performance.  The  effects  of  both  of  these  modifications  on  pilot  opinion  and 
performance  were  then  compared  to  the  baseline  system. 

The  third  and  final  phase  focused  on  this  comparison  between  the  LAMARS 
optimized  system  developed  in  phase  two  and  the  baseline  STAV  control  system.  The 
optimized  system  was  tested  by  all  three  pilots  to  ensure  that  they  agreed  that  the  chosen 
values  for  force  gradient,  spoiler  retraction,  and  gradient  change  were  all  optimal.  All  the 
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pilots  then  retested  the  baseline  system  and  compared  their  results  to  the  previous 
baseline  testing  to  ensure  that  any  improvement  in  pilot  opinion  or  performance  could  not 
be  attributed  to  practice  alone.  A  TPS  staff  pilot  then  flew  both  the  baseline  and 
optimized  system  in  order  to  evaluate  any  differences  between  the  two  systems  and 
corroborate  or  refute  the  test  team  results.  The  flight  test  engineers  and  flight  test  weapon 
systems  officer  then  flew  to  familiarize  themselves  with  what  the  pilots  were  feeling  and 
to  practice  the  test  procedures  to  be  used  during  flight  testing. 

The  data  analysis  plan  for  the  LAMARS  testing  was  created  by  the  TMP  team 
prior  to  actual  simulator  testing,  ft  sought  to  begin  the  data  analysis  concurrently  with 
testing,  so  that  the  TMP  team  could  adapt  if  the  testing  was  not  proceeding  according  to 
plan.  This  method  was  used  to  provide  the  most  flexibility  to  the  test  effort,  a  crucial 
factor  when  dealing  with  a  set  test  schedule.  While  at  the  LAMARS  facility,  copies  were 
made  of  both  the  parametric  data  for  each  run  as  well  as  any  audio  or  video  recordings 
that  were  noted  by  the  test  conductor  as  particularly  interesting.  Each  data  run  was  given 
a  number  and  a  hard  copy  of  a  test  card  was  used  by  the  test  conductor  to  record  both 
pilot  comments  and  initial  performance  parameters.  A  run  number  for  all  the 
programmed  test  inputs  and  semi-closed-loop  maneuvers  was  also  recorded  during  the 
flying  qualities  portion  of  testing.  At  LAMARS,  a  DVD  of  all  the  recorded  parameters 
for  each  test  run  was  made.  While  testing,  excel  spreadsheets  were  created  to  input 
Cooper-Harper  ratings  and  performance  data  in  order  to  get  a  real  time  quick-look  of 
trend  data  on  how  the  testing  was  proceeding.  After  LAMARS  testing  was  completed,  a 
brief  was  conducted  to  summarize  the  quick-look  results  and  gather  any  preliminary 
lessons  learned. 
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After  returning  to  TPS,  the  data  were  analyzed  in  order  to  detennine  if  the  test 
objectives  were  met.  The  goal  of  the  data  reduction  after  LAMARS  testing  was  to 
establish  a  data  set  to  compare  to  TIFS  testing  and  to  prepare  Matlab,  Excel,  and  other 
data  reduction  techniques  to  streamline  the  effort  when  reducing  TIFS  data.  For  the  first 
phase  of  testing,  the  flying  qualities  of  the  STAV  model  as  implemented  on  LAMARS 
were  analyzed.  Additionally,  the  alpha,  gamma,  and  q-command  control  systems  were 
compared.  For  the  second  phase,  the  results  from  the  model  optimization  were  laid  out. 
This  included  looking  at  the  improvement  in  CHR  as  well  as  performance,  and  linking 
this  improvement  with  the  pilot  comments.  The  analysis  of  the  optimization  sought  to 
explain  the  reasons  for  the  improvement.  The  results  from  the  repeat  testing  of  the 
baseline  model  were  then  analyzed  to  uncover  any  learning  trends  in  the  data.  For  the 
third  phase,  comparisons  between  the  baseline  and  optimized  system  were  made  by 
plotting  pilot  aggressiveness  and  duty  factor,  as  well  as  histograms  of  CHR  for  each 
system. 

3.6  TIFS  Flight  Testing 

Flight  testing  of  the  STAV  model  was  conducted  on  the  NC-131H  Total  In-Flight 
Simulator,  a  six  degree  of  freedom  in-flight  simulator  operated  by  Calspan.  The  flight 
test  sorties  were  accomplished  from  10-13  September  2007  in  the  airborne  traffic  pattern 
at  Niagara  Falls  International  Airport.  The  goal  of  flight  testing  was  to  meet  all  three  of 
the  test  management  project  (TMP)  team  objectives:  determine  the  powered  approach 
handling  qualities  of  the  baseline  STAV  model,  compare  the  LAMARS  optimized  control 
system  to  the  baseline  STAV  control  system,  and  detennine  the  flying  qualities  for  the 
TIFS  simulation  of  the  STAV  flight  control  system.  The  primary  objective  for  this  thesis 
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was  to  evaluate  the  handling  qualities  of  the  STAV  flight  control  system  model  during 
the  powered  approach  phase  of  flight,  an  objective  supported  by  the  three  TMP 
objectives.  Cooper-Harper  ratings  were  the  primary  evaluation  metric  for  the  flight  tests, 
and  were  described  in  more  detail  in  Chapter  2.  The  desired  and  adequate  performance 
criteria  were  developed  by  the  test  team  in  conjunction  with  the  model  developer  based 
on  previous  experience  and  expected  design  limitations.  In  addition  to  a  CHR,  a  Pilot  In- 
the-loop  Oscillation  rating  was  given  by  the  pilot  if  a  PIO  was  encountered  during  the 
approach  and  landing  task.  If  a  PIO  was  encountered,  the  pilot  rated  it  according  to  the 
scale  and  provided  comments  on  how  objectionable  the  motion  was  and  what  effect  it  had 
on  pilot  opinion.  The  PIOR  was  used  as  another  measure  of  perfonnance  in  detennining 
the  handling  qualities  of  the  STAV  model.  A  description  of  this  scale  was  in  Chapter  2. 

The  TIFS  test  plan  began  with  the  test  methods  and  procedures  conducted  during 
LAMARS  testing  and  refined  them  as  necessary  to  make  the  flight  testing  flow  more 
efficiently.  The  factorial  design  method  used  in  LAMARS  testing  included  four 
variables:  pilot,  offset,  crosswind,  and  approach  airspeed.  This  matrix  was  executed  on 
LAMARS  to  verily  predictions  and  narrowed  down  the  actual  TIFS  flight  test  matrix. 
The  TIFS  flight  test  matrix  also  had  four  variables,  but  instead  of  approach  airspeed  as 
the  fourth  variable,  in  flight  testing  the  final  variable  was  the  control  system,  either  the 
baseline  STAV  model  or  the  LAMARS  optimized  system.  An  approach  airspeed  of  185 
knots  was  selected  as  optimal  during  LAMARS  testing,  and  was  no  longer  a  variable. 

The  TIFS  test  plan  also  drew  on  the  experiences  of  the  Calspan  pilots  and 
engineers  who  had  conducted  other  flight  tests  on  TIFS.  The  TMP  team  looked  at  the 
process  of  using  the  Variable  Stability  System  (VSS),  and  how  to  use  it  most  effectively. 
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Previous  flight  test  programs  on  TIFS  indicated  that  the  optimal  time  to  switch  to  the 
VSS  and  transfer  control  to  the  evaluation  pilot  was  on  downwind.  This  procedure 
allowed  pilots  to  gain  an  initial  feel  of  the  system  while  turning  base  and  final,  prior  to 
conducting  the  approach  and  landing  tasks.  Discussions  with  Calspan  also  revealed  that 
the  maximum  TIFS  sortie  duration  was  two  hours.  This  drove  the  design  of  the  test 
matrix  to  make  the  most  efficient  use  of  flight  time  by  maximizing  the  number  of 
approaches  flown  on  each  of  the  five  planned  flights. 

The  flight  testing  used  TIFS-generated  localizer  and  glideslope  information  to 
ensure  repeatability  in  task  performance  between  the  different  test  pilots.  This  procedure 
was  essential  during  the  lateral-offset  tasks,  where  a  consistent  offset  point  was  required. 
This  TIFS  capability,  which  used  the  global  positioning  system,  also  allowed  the  test 
team  to  shift  the  desired  touchdown  point  to  1,500  feet  down  the  runway,  a  point  on  the 
runway  which  allowed  better  threshold  clearance  and  enhanced  test  safety.  The  TIFS 
allowed  the  team  to  capture  “touchdown”  parameters  at  an  actual  altitude  of  20  feet  AGL, 
since  landing  gear  airspeed  restrictions  limited  testing  to  low  approaches  only.  The 
landing  distance  criteria  were  measured  from  this  “touchdown”  point.  These  planned  low 
approaches  not  only  allowed  the  TIFS  to  conduct  gear  down  approaches  at  speeds  above 
maximum  wheel  touchdown  speed,  but  allowed  the  test  to  model  the  pilot  eye  height  of 
the  STAV.  When  passing  through  the  point  on  the  touchdown  plane,  the  performance 
parameters  were  recorded  and  displayed  to  the  test  team  so  that  a  Cooper-Harper 
evaluation  could  be  completed. 

In  the  weeks  prior  to  flight  testing,  Calspan  conducted  one  functional  check  flight 
and  two  calibration  flights  at  the  direction  of  the  TMP  team.  The  functional  check  flight 
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ensured  that  the  TIFS  aircraft  would  be  ready  for  flight  testing  after  several  years  spent  in 
“flyable  storage”.  The  calibration  flights  integrated  the  STAV  model  with  the  VSS  on 
TIFS,  a  task  made  more  complex  by  the  fact  that  the  STAV  model  required  that  an 
additional  computer  be  brought  aboard  the  aircraft  in  order  to  run  the  flight  tests  properly. 
No  modifications  were  made  to  the  STAV  model  itself;  all  changes  included  just  the  top- 
level  wrapper  around  the  STAV  model.  The  additional  computer  was 
VxWorks/PowerPC-based,  which  communicated  to  the  model-following  computer  on 
TIFS  via  a  standard  1553  bus.  The  real-time  model  was  implemented  on  TIFS  with  the 
VxWorks  program,  which  was  an  identical  environment  as  Linux  but  included  a  gcc/g++ 
compiler.  The  source  code  and  make-file  which  were  originally  compiled  and  checked  in 
the  Linux/Unix  environment  during  previous  testing  therefore  also  worked  in  TIFS.  In 
addition  to  the  model  calibration  and  integration,  a  TPS  instructor  ensured  that  all  the 
various  safety  trips  aboard  the  aircraft  were  operational  prior  to  test  team  arrival.  The 
pilot  ran  through  the  flight  test  cards  to  ensure  that  all  maneuvers  were  safe  and  that  all 
parameters  were  being  recorded  and  displayed  correctly.  Finally,  the  pilot  made  sure  that 
there  were  no  significant  time  delays  in  the  system  that  would  impact  testing,  and  that  the 
TIFS  model  following  performance  was  satisfactory.  These  checks  of  the  time  delay  and 
model  following  were  performed  by  running  a  predetermined  set  of  test  team 
Programmed  Test  Inputs  (PTI)  through  the  STAV  model  as  implemented  on  TIFS  and 
analyzing  the  response.  These  preparation  flights  were  conducted  the  week  prior  to  flight 
testing.  The  flight  test  cards  are  located  in  appendix  C  in  figures  C-10  through  C-13,  and 
have  the  same  tasks  as  the  previous  test  cards  except  for  the  vertical  offset  task,  which 
was  not  accomplished  during  flight  testing.  The  performance  criteria  are  also  the  same 
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except  for  the  landing  zone,  which  was  increased  in  size  following  inputs  from 
operational  bomber  pilots,  and  the  touchdown  airspeed,  which  was  removed. 

Flight  testing  planned  to  fly  one  sortie  the  first  day  of  flight  test,  and  then  two 
sorties  each  of  the  following  two  days.  This  allowed  the  test  team  to  delay  the  flight  tests 
if  the  weather  was  not  sufficient  or  if  there  were  maintenance  or  technical  issues.  It  also 
allowed  the  data  from  a  test  flight  to  be  analyzed  immediately  after  landing,  so  that  any 
lessons  learned  could  be  applied  to  the  subsequent  flights.  Calspan  pilots  performed  the 
initial  taxi  and  take  off,  and  flew  the  TIFS  in  between  each  run  while  the  evaluation  pilot 
(test  team  test  pilot)  was  working  with  the  test  conductor  to  assign  a  Cooper-Harper 
rating.  The  test  runs  commenced  once  aircraft  control  had  been  transferred  to  the 
evaluation  pilot.  The  evaluation  pilot  assumed  control  and  performed  the  required  task. 
Each  evaluation  pilot  began  the  sequence  of  test  points  with  a  nominal  or  baseline 
precision  approach  and  landing.  To  increase  pilot  workload,  the  crosswinds  were 
increased  to  seven  knots  and  the  approach  was  repeated.  The  pilot  then  flew  an  offset 
approach  with  seven  knots  of  crosswind.  Each  point  was  terminated  by  either  a 
simulated  touchdown,  a  safety  pilot  tennination,  or  via  the  safety  trips  in  the  variable 
stability  system  onboard  the  TIFS. 

When  the  aircraft  was  on  the  downwind  leg,  at  approximately  1500  feet  AGL,  the 
evaluator  pilot  took  control  of  the  aircraft  and  performed  a  series  of  programmed  test 
inputs  and  semi-closed-loop  tasks.  These  inputs  included  steps  and  doublets  in  the  pitch 
and  yaw  axes,  as  well  as  a  step  in  the  roll  axis.  The  pilot  recovered  the  aircraft  to  level 
flight  after  directed  by  the  Calspan  engineer  in  the  back  of  the  aircraft.  The  pilot  then 
performed  low  gain  capture  tasks  in  pitch,  roll,  and  heading.  All  maneuvers  and 
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programmed  test  inputs  were  repeated  with  the  spoilers  completely  retracted,  and  a  set  of 
pitch  steps  were  accomplished  while  the  spoilers  were  being  retracted. 

For  all  approaches,  the  TIFS  generated  a  2.5  degree  glide  slope  that  aimed  at  a 
point  750  feet  long  of  the  runway  threshold.  This  point  was  chosen  to  provide  sufficient 
safety  clearance  with  a  road  that  crossed  perpendicular  to  the  runway  just  prior  to  the 
overrun.  This  provided  a  ground  distance  of  approximately  750  feet  to  flare  before  the 
planned  touchdown  point  at  1,500  feet  long  of  the  runway  threshold.  The  desired  aim 
point  and  touchdown  point  are  shown  in  figure  27. 


Figure  27  -  Desired  Aim  Point  and  Touchdown  Point 

For  all  tasks  requiring  crosswinds,  the  TIFS  side  force  generators  were  used  to 
simulate  a  crosswind.  The  TIFS  briefed  capabilities  stated  that  the  side  force  generators 
could  negate  up  to  a  fifteen  knot  actual  crosswind,  or  add  to  the  actual  crosswinds  to 
generate  the  effect  of  a  fifteen  knot  crosswind.  During  flight  testing,  the  test  team  found 
that  when  TIFS  generated  an  effective  crosswind  greater  than  seven  knots,  the  variable 
stability  system  was  prone  to  nuisance  systems  trips  with  normal  pilot  inputs.  These  trips 
were  due  to  the  hinge  forces  generated  by  the  side  force  controllers  at  a  nominal  approach 
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speed  of  185  knots.  Therefore,  TIFS  was  used  to  generate  or  eliminate  a  maximum 
crosswind  of  seven  knots. 

For  normal  landing  tasks,  the  2.5  degree  glide  slope  was  aligned  with  the 
centerline.  For  the  lateral  offset  tasks,  the  glide  slope  was  offset  by  200  feet  from 
centerline,  as  shown  in  figure  28.  It  could  be  offset  either  right  of  left,  based  on  the 
lateral  correction  direction  dictated  by  the  actual  crosswinds.  Any  generated  crosswinds 
required  were  from  the  direction  opposite  of  the  offset,  which  increased  the  task  difficulty 
by  forcing  the  pilot  to  correct  into  the  crosswind.  In  the  cockpit,  the  glideslope 
presentation  to  the  pilot  indicated  on  course  when  the  pilot  was  lined  up  on  the  200  foot 
lateral  offset  point.  At  300  feet  AGL,  the  test  conductor  called  “maneuver”,  and  the  pilot 
aggressively  maneuvered  back  to  the  centerline  for  the  lateral  offset  tasks,  in  an  effort  to 
land  at  the  desired  touchdown  point,  which  remained  the  same  as  the  normal  landing  task. 
The  approach  airspeed  was  185  knots  in  all  cases. 


Figure  28  -  Lateral  Offset  Points 

The  data  analysis  plan  used  in  reducing  and  analyzing  the  TIFS  flight  test  data 
followed  the  same  process  used  for  the  LAMARS  data.  While  at  the  Calspan  facility  in 
Niagara  Falls,  copies  were  made  of  both  the  parametric  data  for  each  run  as  well  as  any 
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audio  or  video  recordings.  Each  data  run  was  given  a  number,  so  that  it  could  be  more 
easily  organized  after  testing  was  complete.  On  each  data  run,  a  hard  copy  of  a  test  card 
was  used  by  the  test  conductor  to  record  both  pilot  comments  and  initial  performance 
parameters.  During  testing,  test  team  members  created  excel  spreadsheets  to  input 
Cooper-Harper  ratings  and  performance  data  in  order  to  get  a  real  time  quick-look  of 
trend  data  on  how  the  testing  was  proceeding.  When  the  test  team  returned  to  Test  Pilot 
School  (TPS),  the  data  were  analyzed  in  order  to  determine  whether  each  objective  was 
met.  The  LAMARS  data  reduction  set  a  baseline  for  the  TIFS  testing  and  prepared  the 
Matlab,  Excel,  and  other  data  reduction  techniques  that  streamlined  the  TIFS  data 
reduction  effort. 

At  Calspan,  a  DVD  of  all  the  recorded  in-flight  parameters  for  each  flight  was 
made.  TIFS  also  had  a  video  camera  in  the  evaluation  cockpit  to  record  an  over  the 
pilot’s  shoulder  view  of  the  testing.  DVDs  from  each  flight  were  gathered  by  the  test 
team.  During  each  flight,  the  test  conductor  again  recorded  pilot  comments  and  initial 
parameters  on  a  hard  copy  of  each  test  card,  which  were  marked  with  a  run  number.  A 
run  number  for  all  the  programmed  test  inputs  and  semi-closed-loop  maneuvers  was  also 
recorded.  After  each  flight,  the  pilot  summarized  their  comments  on  the  flight  and  wrote 
them  in  a  daily  flight  test  report.  This  daily  flight  test  report  included  lessons  learned  in 
testing  that  would  aid  the  subsequent  pilots  and  test  conductors  in  their  data  flights. 
Cooper-Harper  ratings  and  perfonnance  infonnation  were  again  inputted  into  an  Excel 
spreadsheet,  to  provide  a  quick-look  on  trend  data.  This  process  continued  between  each 
flight.  After  flight  testing  was  completed,  a  brief  with  Calspan  was  conducted  to 
summarize  the  quick-look  results  and  gather  any  preliminary  lessons  learned. 
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After  returning  to  TPS,  the  flight  test  engineers  took  the  data  and  reduced  it 
according  to  each  test  team  objective.  For  the  first  objective,  Cooper-Harper  ratings  of 
the  baseline  system  were  summarized  on  a  histogram  according  to  both  task  and 
individual  pilot.  For  the  second  objective,  Cooper-Harper  ratings  for  both  the  baseline 
and  optimized  system  were  compared  according  to  both  task  and  pilot.  Pilot  perfonnance 
using  both  of  the  systems  was  also  compared.  Another  comparison  between  the  baseline 
and  optimized  system  was  made  by  plotting  pilot  aggressiveness  and  duty  factor.  For  the 
third  objective,  the  model  following  capability  of  the  TIFS  was  displayed.  This  included 
flight  conditions  with  both  calm  conditions  and  with  turbulence.  An  additional  method 
used  to  investigate  the  STAV  handling  qualities  measured  pilot  aggressiveness  and  duty 
factor  when  conducting  the  different  approach  and  landing  tasks.  Pilot  aggressiveness 
was  determined  by  measuring  the  speed  of  the  inceptor  movements,  while  duty  factor 
was  a  measure  of  the  percentage  of  time  the  pilot  was  “in-the-loop”,  moving  the  inceptor. 
This  method  was  used  post-flight  to  compare  the  pilot’s  perception  of  workload  and 
predictability  during  the  tasks  with  the  actual  inceptor  movements. 

3.7  Summary 

This  chapter  explained  in  detail  the  various  test  methods  and  procedures  used 
during  the  course  of  this  thesis.  It  first  focused  on  the  scope  and  assumptions  of  this 
thesis.  It  then  covered  the  overall  general  test  methodology,  including  the  initial  test 
procedures  developed  using  the  previous  LAMARS  testing  by  the  NGC.  The  methods 
and  procedures  used  during  each  of  the  three  different  test  sections  were  then  outlined, 
including  a  data  analysis  plan  for  the  results  of  each  section.  As  the  testing  progressed, 
the  methods  and  procedures  were  modified  not  only  to  fit  the  new  test  environment,  but 
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also  to  improve  the  flow  and  management  of  data.  The  lessons  learned  from  a  previous 
section’s  testing  were  applied  to  the  next  and  so  on;  resulting  in  testing  that  became  more 
refined  and  efficient  as  it  progressed.  This  evolution  in  testing  applied  not  only  to  the 
conduct  of  the  test,  but  also  to  the  data  reduction  at  the  conclusion  of  testing. 
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4.0  Results  and  Analysis 


4.1  Overview 

This  chapter  contains  the  results  and  analysis  of  all  testing  conducted  throughout 
this  thesis,  and  is  divided  into  the  three  main  test  sections:  Infinity  Cube  Simulator  (ICS) 
testing,  Large  Amplitude  Multi-mode  Aerospace  Research  Simulator  (LAMARS)  testing, 
and  Total  In-Flight  Simulator  (TIFS)  testing.  For  the  ICS  testing,  it  summarizes  the  pool 
of  pilots  by  number  and  classification.  It  breaks  down  the  results  of  each  test  section  first 
by  aircraft,  then  by  overall  handling  qualities  (HQ)  rating  and  data  precision,  and  finally 
by  HQ  rating  and  data  precision  according  to  pilot  classification.  For  the  LAMARS  and 
TIFS  testing,  it  looks  at  results  of  the  baseline  and  LAMARS  optimized  models,  as  well 
as  the  comparison  between  the  two.  The  results  include  pilot  performance  and  CHR  for 
all  three  test  sections  and  pilot  workload  vs.  aggressiveness  for  the  LAMARS  and  TIFS 
testing.  Each  section  discusses:  if  the  pilot  ratings  differed  according  to  classification; 
ways  to  improve  the  test  results;  and  underlying  issues  that  hindered  the  tests  or  proved  to 
be  poor  assumptions. 

4.2  Infinity  Cube  Simulator  Testing 

Testing  in  the  Infinity  Cube  Simulator  took  place  from  16-18  October,  2006. 
Testing  followed  the  procedures  and  methods  explained  in  the  previous  chapter.  After 
submitting  a  request  to  the  pilot  population  at  the  Air  Force  Institute  of  Technology, 
nineteen  pilots  were  available  to  participate  in  the  tests.  These  nineteen  pilots  had 
varying  backgrounds  and  experience  levels.  This  pool  of  pilots  averaged  over  1,570 
hours  of  flight  time  each  in  thirteen  different  fixed  wing  and  rotary  wing  aircraft.  The 
following  table  5  shows  the  pilot  pool  for  the  ICS  testing,  including  total  number  and 
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average  flight  time  of  each  pilot  group.  Table  G-l  in  appendix  G  contains  individual 
pilot  information. 


Table  5  -  Infinity  Cube  Simulator  Pilot  Pool 


Pilot  Group 

USAF 

Navy 

Civilian 

Fighter 

Heavy 

Non-Test 

Test 

Number 

12 

6 

1 

10 

9 

16 

3 

Avg  Time  (Hrs) 

1647 

1342 

2000 

1542 

1611 

1495 

2000 

The  pilots  conducted  228  total  approaches  and  landings,  including  57  for  practice 
and  171  for  data.  Each  pilot  flew  twelve  approaches,  three  for  practice  and  nine  for  data. 
This  further  broke  down  into  one  practice  and  three  data  runs  each  for  the  T-38  model, 
the  STAV  model  at  175  knots,  and  the  STAV  model  at  195  knots.  Four  tasks  were 
accomplished  during  testing,  a  precision  approach  and  a  normal  landing  on  the  first  run,  a 
lateral  offset  landing  on  the  second  run,  and  a  vertical  offset  landing  on  the  third  run. 
The  test  cards  in  appendix  C  provide  more  detail  on  each  task,  and  the  approach  and 
landing  perfonnance  criteria  are  displayed  in  tables  6  and  7. 

Table  6  -  Infinity  Cube  Simulator  Approach  Criteria 


Precision  Approach 

Desired 

Adequate 

Deviation  from  approach  airspeed 

±5  knots 

±10  knots 

Deviation  from  glideslope 

±  0.5  dot 

±  1.0  dot 

Deviation  from  localizer 

±  0.5  dot 

±  1.0  dot 

Table  7  -  Infinity  Cube  Simulator  Landing  Criteria 


Precision/  Offset  Landings 

Desired 

Adequate 

Landing  zone 

±25  ft  laterally 
±500  ft  longitudinally 

±50  ft  laterally 
±1000  ft  longitudinally 

Deviation  from  touchdown  airspeed 

±  5  knots 

±10  knots 

Max  bank  angle  below  50  feet 

±  5  degrees 

±  7  degrees 

Max  touchdown  sink  rate 

6  ft/sec 

10  ft/sec 

Deviation  from  runway  heading  at 
touchdown 

±  2  degrees 

±  4  degrees 
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4.2.1  T-38 


The  T-38  model  was  flown  first  by  every  pilot.  After  completing  each 
data  run  and  analyzing  their  performance  and  workload,  the  pilot  would  give  a 
longitudinal  and  lateral  CHR  for  each  task.  The  average  CHR  for  the  T-38  tasks  was  a 
three,  corresponding  to  level  one  HQ.  The  original  testing  on  the  T-38  was  completed 
before  the  CHR  scale  came  into  existence,  so  there  is  no  exact  historical  comparison. 
However,  the  USAF  policy  on  aircraft  HQ  states  that  for  nonnal  mission  tasks,  the  HQ 
should  be  level  one.  The  T-38  has  been  flying  operationally  in  the  USAF  for  the  past 
forty-six  years,  and  although  it  can  be  tricky  to  land,  the  HQ  are  generally  accepted  as 
level  one  for  approach  and  landing.  Therefore,  the  level  one  rating  given  by  the  ICS  test 
pilots  corresponded  well  with  real-world  operational  experience.  There  were  no 
statistically  significant  CHR  or  performance  differences  between  any  of  the  pilot  groups 
for  the  T-38  testing.  The  largest  differences  in  longitudinal  and  lateral  CHR  were 
between  the  Air  Force  and  Navy  pilots  (figure  29),  while  the  greatest  difference  in 
performance  achieved  was  between  fighter  and  heavy  pilots  (figure  30).  The  use  of  a 
baseline  aircraft  was  vital  to  ensure  that  the  pilots  were  correctly  using  the  CHR  scale.  It 
served  as  a  basis  by  which  the  results  of  a  group  of  non-test  pilots  could  be  compared  to 
historical  data. 
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Figure  29  -  Air  Force  vs.  Navy  T-38  CHR 


Figure  30  -  Fighter  vs.  Heavy  T-38  Performance 
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The  performance  achieved  by  each  pilot  was  termed  parameter  accuracy,  and  was 
calculated  by  weighting  each  performance  criterion  equally  and  then  adding  the  total 
deviations  from  ideal  touchdown  parameters  (on  speed,  heart  of  the  landing  zone,  no 
bank  or  heading  deviations,  zero  sink  rate).  All  plots  were  formed  by  calculating  the 
mean  and  standard  deviation  of  each  pilot  group,  and  then  taking  a  normal  distribution  of 
the  data.  A  summary  of  the  CHR  for  each  task  and  model  is  located  in  table  A-l  in 
appendix  A. 

4.2.2  STAV  (ICS) 

The  version  1  STAV  model  was  tested  next,  and  included  the  30%  spoiler 
bias  mentioned  previously.  Half  of  the  pilots  flew  the  test  tasks  at  175  knots  approach 
speed  first,  and  then  at  195  knots.  The  other  half  flew  in  the  reverse  order.  The  ratings  of 
both  of  these  groups  were  analyzed  to  detennine  how  much  the  ratings  improved  from 
the  first  set  of  approaches  to  the  second  set  of  approaches.  The  average  CHR  improved 
0.65  for  the  175  to  195  group,  and  got  worse  by  0.21  for  the  195  to  175  group.  These 
values  were  used  to  determine  the  mean  learning  effect,  which  was  applied  to  the  data 
from  both  groups  to  cancel  out  any  perceived  ratings  improvement  caused  solely  by 
learning  (i.e.  the  pilots  performing  better  as  they  fly  the  STAV  more).  This  allowed  the 
175  knot  and  195  knot  models  to  be  compared  by  all  pilots  equally,  indeterminate  of  test 
run  order.  Overall,  the  average  CHR  was  5.2  for  the  175  knot  STAV  and  4.9  for  the  195 
knot  STAV,  a  statistically  insignificant  ratings  difference. 

The  mean  longitudinal  and  lateral  learning  effects  were  0.43  and  0.62,  which 
meant  that  the  CHR  of  whatever  a  pilot  flew  second  improved  by  that  amount.  After 
applying  these  learning  effects  to  the  data,  the  effects  themselves  were  analyzed  to 
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determine  if  one  group  of  pilots  learned  at  a  different  rate  than  another.  As  expected, 
individual  pilots  learned  at  different  rates.  When  the  learning  rates  of  different  pilot 
groups  were  studied,  some  interesting  trends  broke  out.  Although  the  statistical 
difference  between  pilot  groups  was  lessened  after  taking  variation  among  the  nineteen 
different  pilots  into  account,  the  mean  learning  effects  of  each  group  depicted  some 
disparity.  Figure  31  shows  the  mean  longitudinal  learning  effects  of  each  pilot  group. 
The  largest  differences  in  learning  effect  were  in  the  longitudinal  realm,  where  both  Air 
Force  vs.  Navy  pilots  and  Fighter  vs.  Heavy  pilots  showed  opposite  learning  trends.  The 
Air  Force  and  Fighter  pilot  groups  tended  to  rank  better  whatever  STAV  approach  speed 
they  tested  second.  The  Navy  and  Heavy  pilot  groups  tended  to  rank  whatever  STAV 
approach  speed  they  tested  first  as  slightly  better. 


Data  Basis:  ICS  Testing 
Config:  STAV  Version  1 
Test  Dates:  16-18  Oct  2006 
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Figure  31  -  Mean  Longitudinal  Learning  Effect  in  ICS  Testing 
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In  the  lateral  realm,  the  learning  effect  differences  were  not  as  significant.  All 
pilot  groups  tended  to  rank  better  whatever  STAV  approach  speed  they  tested  second. 
Figure  32  shows  the  mean  lateral  learning  effects  of  each  pilot  group. 


-1.4 
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Data  Basis:  ICS  Testing 


Config:  STAV  Version  1 
Test  Dates:  16- 18  Oct  2006 
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Figure  32  -  Mean  Lateral  Learning  Effect  in  ICS  Testing 
The  overall  results  for  the  STAV  model  were  that  every  task  at  both  approach 
speeds  was  rated  level  two.  This  compared  closely  with  the  results  from  the  initial  NGC 
LAMARS  testing,  where  every  task  at  both  approach  speeds  was  also  rated  level  two, 
except  for  the  vertical  offset  landing  at  195  knots,  which  was  rated  level  one.  Table  8 
shows  the  average  CHR  and  standard  deviation  for  each  task  for  both  the  initial  NGC 
LAMARS  testing  and  the  ICS  testing.  Even  though  the  test  pilot  sample  size  increased 
by  a  factor  greater  than  six,  the  standard  deviation  for  each  task  remained  the  same  order 
of  magnitude. 
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Table  8  -  Initial  NGC  LAMARS  Testing  vs.  ICS  Testing  CHR 


Task 

Initial  Mean 

Initial  a 

ICS  Mean 

ICS  CT 

Approach  - 175 

4.67 

0.58 

4.26 

1.71 

Approach  - 195 

4.50 

0.50 

4.07 

1.40 

Land  - 175 

4.50 

0.50 

5.59 

1.79 

Land  -  195 

4.33 

1.15 

5.00 

1.74 

Lateral  Offset  -  175 

4.50 

0.71 

5.50 

1.81 

Lateral  Offset  -  195 

4.67 

1.53 

5.45 

1.59 

Vertical  Offset  -  175 

4.33 

1.15 

5.28 

1.83 

Vertical  Offset  -  195 

3.33 

1.33 

5.26 

1.64 

The  aerodynamic  characteristics  of  the  version  1  STAV  model  were  used  to 
calculate  the  predicted  HQ.  This  resulted  in  predicted  HQ  of  level  one  or  two.  As  shown 
before,  the  HQ  were  rated  level  two  during  ICS  testing.  Table  9  shows  the  predicted  HQ 
based  off  of  the  aerodynamic  characteristics  of  the  version  1  STAV  model. 


Table  9  -  STAV  Aerodynamic  Characteristic  Predicted  HQ 


Characteristic 

STAV  value 

Predicted  HQ  level 

Csp 

1.85  -  1.92 

2 

®sp 

1.4  -  2.7 

1 

Cp 

0-0.13 

1/2 

CAP  (G)sp2/(n/a)) 

0.48  - 1.02 

N/A 

n/a 

4.09-7.16 

1 

cosp  vs.  n/a 

N/A 

1 

CAP  vs.  (;sp 

N/A 

2 

G>spTe2  vs.  ^Sp 

N/A 

2 

There  were  no  statistically  significant  CHR  or  performance  differences  between 
any  of  the  pilot  groups  for  the  195  knots  STAV  testing.  The  largest  difference  in  CHR 
was  between  Non-test  and  Test  pilots  (figure  33). 
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Figure  33  -  Non-test  vs.  Test  STAV  195  Longitudinal  CHR 
For  the  175  knot  STAV  model,  there  were  statistically  significant  differences 
between  Air  Force  vs.  Navy  pilots  and  Non-Test  vs.  Test  pilots.  More  than  68%  of  the 
Navy  pilots  rated  the  175  knot  STAV  model  better  than  the  195  knot  model,  and  the 
reverse  corresponded  to  Air  Force  Pilot  ratings.  Figure  34  shows  the  longitudinal  CFIR 
differences  between  Air  Force  and  Navy  pilots.  These  differences  indicated  a  Navy  pilot 
preference  for  the  slower  speed  approaches,  a  fact  that  matched  well  with  current  naval 
approach  operations,  which  are  conducted  at  lower  airspeeds  on  the  back  side  of  the 
power  curve. 
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Figure  34  -  Air  Force  vs.  Navy  STAV  175  Longitudinal  CHR 
When  test  and  non-test  pilot  longitudinal  and  lateral  CHR  were  compared  for  the 

175  knot  STAV  model,  the  differences  were  even  greater.  Over  74  %  of  non-test  pilots 
rated  the  175  knot  STAV  model  better  than  the  test  pilots  for  longitudinal  CHR,  and  over 
78%  for  lateral  CHR.  The  following  figures  35  and  36  clearly  depict  these  statistically 
significant  differences  in  both  longitudinal  and  lateral  CHR.  The  differences  between 
these  two  groups  are  most  likely  the  result  of  improper  use  of  the  CHR  scale  than  a 
preference  for  a  certain  approach  speed.  Test  pilots  are  more  apt  to  rate  an  aircraft 
properly  based  on  workload  and  performance.  Although  briefed  on  proper  use  of  the 
CHR  rating  scale,  non-test  pilots  showed  a  potential  tendency  to  rate  the  aircraft  better 
than  what  the  workload  and  performance  called  for,  basing  any  lack  of  performance  more 
on  piloting  skill  than  on  aircraft  deficiencies.  All  of  the  test  pilots  were  also  Air  Force 
pilots,  another  potential  influence  on  the  ratings  differences. 
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Figure  35  -  Non-test  vs.  Test  STAY  175  Longitudinal  CHR 


Figure  36  -  Non-test  vs.  Test  STAV  175  Lateral  CHR 
As  mentioned  previously,  a  mean  learning  effect  was  applied  to  the  data  so  that 
the  175  and  195  knot  STAV  tasks  could  be  isolated  independent  of  test  run  order.  This 
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allowed  the  data  to  be  reduced  to  determine  if  any  pilot  group  preferred  a  certain 
approach  speed  over  another.  The  largest  difference  in  preferred  approach  speed  was 
between  the  Air  Force  and  Navy  pilots.  Of  all  the  Air  Force  pilots,  68%  preferred  the 
195  knot  STAV  approach  speed,  while  62%  of  the  Navy  pilots  preferred  the  175  knot 
approach  speed.  These  results  agreed  with  the  previous  STAV  175  knot  CHR  differences 
shown  earlier  in  this  chapter.  Figure  37  shows  the  preference  differences  between  Air 
Force  and  Navy  pilots.  The  piloting  techniques  employed  by  the  pilots  of  different 
services  showed  that  previous  experience  had  an  impact  on  HQ  rating. 


Figure  37  -  Air  Force  vs.  Navy  Preferred  Approach  Speed 
STAV  parameter  accuracy  (performance)  of  each  pilot  was  calculated  in  the  same 
manner  as  the  T-38  parameter  accuracy,  by  weighting  each  performance  criterion  equally 
and  then  adding  the  total  deviations  from  ideal  touchdown  parameters  (on  speed,  heart  of 
the  landing  zone,  no  hank  or  heading  deviations,  zero  sink  rate).  The  only  two  groups  to 
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show  a  statistically  significant  difference  in  performance  achieved  were  the  Fighter  and 
Heavy  pilots,  and  this  difference  is  depicted  in  figure  38. 


Figure  38  -  Fighter  vs.  Fleavy  Parameter  Accuracy  Achieved 
About  68%  of  the  Fighter  pilots  achieved  more  precise  touchdown  parameters 
than  Heavy  pilots.  These  results  make  sense,  because  fighter  pilots  in  general  have  to  fly 
with  greater  precision  than  heavy  pilots  in  order  to  accomplish  an  operational  mission. 
These  results  did  not  speak  to  the  skill  of  a  certain  pilot  group,  or  say  that  one  group  of 
pilots  was  better  than  another;  it  merely  highlighted  the  fact  that  the  type  of  flying 
normally  conducted  by  each  pilot  group  had  an  impact  on  task  performance. 

Overall,  the  pilot  accuracy  correlated  well  with  the  pilot  rating,  where  the  pilots 
who  performed  the  best  generally  gave  the  best  CHR.  This  was  not  always  the  case, 
since  pilot  workload  was  also  taken  into  account  when  compiling  a  CHR,  but  it  was  the 
general  trend.  This  analysis  was  made  to  ensure  that  pilots  who  were  performing  poorly 
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were  not  giving  erroneously  good  CHR.  Figure  39  shows  a  plot  of  pilot  CHR  and 
parameter  accuracy  according  to  each  individual.  The  parameter  accuracy  was  scaled  to 
better  fit  the  plot,  as  the  purpose  was  to  convey  the  accuracy  level  in  relation  to  the  other 
pilots;  the  actual  individual  accuracy  values  were  not  important. 


STAV  Pilot  CHR  vs.  Accuracy 


Data  Basis:  ICS  Testing 
Config:  STAV  Version  1 
Test  Dates:  16-18  Oct  2006 


I - 

- Pilot  Average  Rating 

Pilot  Accuracy 


Figure  39  -  STAV  Pilot  CHR  vs.  Accuracy 

After  analyzing  the  results,  some  areas  for  improvement  and  possible  underlying 
impacts  on  testing  were  postulated.  The  use  of  both  the  heads  up  display  (HUD)  and 
side-stick  caused  some  initial  consternation  with  pilots  not  used  flying  with  either,  but 
this  impact  was  lessened  by  letting  the  pilots  have  a  practice  approach  in  each  model. 
Some  of  the  pilots  complained  that  the  simulator  brightness  hindered  the  visual 
corrections  during  the  offset  landing  tasks  and  during  the  flare.  Having  motion  along 
with  brighter  visuals  would  improve  the  quality  of  the  simulation.  The  STAV  model 
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could  not  be  trimmed,  a  factor  which  impacted  pilot  performance  and  CHR.  The  lack  of 
ground  effect  also  led  to  a  tendency  to  balloon  in  the  flare.  These  impacts  would  be 
mitigated  with  an  increased  fidelity  STAV  model  that  was  trimmable  and  accounted  for 
ground  effect  and  gear  modeling.  The  touchdown  airspeed  criterion  of  160  knots  may 
have  also  had  a  negative  impact  on  CHR,  and  should  be  increased  for  subsequent  testing. 
There  should  only  be  one  CHR  assigned  per  task,  as  it  was  difficult  to  divide  lateral  and 
longitudinal  performance  and  workload  and  assign  a  CHR  for  each. 

The  displayed  HQ  of  the  STAV  model  illustrated  the  need  for  a  thorough  safety 
plan  prior  to  any  flight  testing.  The  speeds  and  offset  tasks  need  to  be  evaluated  to  ensure 
that  all  safety  of  flight  issues  are  met,  and  an  altitude  buffer  between  the  ground  and  the 
aircraft  would  provide  an  extra  margin  of  safety  should  some  of  the  more  disagreeable 
handling  qualities  surface.  Pilots  should  be  allowed  to  conduct  more  approaches,  so  that 
any  learning  effects  can  take  place  prior  to  data  collection.  An  expanded  test  profile 
should  include  not  only  approach  and  landing  tasks,  but  also  tasks  throughout  the 
expected  mission  envelope.  This  testing  should  include  synthetic  vision  testing,  as  some 
sort  of  synthetic  vision  will  be  necessary  to  safely  operate  and  land  the  STAV.  A  HUD 
should  be  used  in  further  testing,  as  it  reduced  pilot  workload,  especially  during  the  flare. 
Further  testing  should  focus  on  using  test  pilots.  These  pilots  do  not  require  similar 
backgrounds;  they  should  just  be  test  pilots  who  are  familiar  with  conducting  a  handling 
qualities  evaluation.  Further  testing  should  be  accomplished  on  some  type  of  aircraft 
with  a  variable  stability  system.  This  would  allow  the  tester  to  look  at  current  and  future 
STAV  models,  as  well  as  the  ability  to  revert  to  another  aircraft  should  the  need  arise 
when  in  close  proximity  to  the  ground. 
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4.3  LAMARS  Testing 

Prior  to  testing  in  LAMARS,  each  HAVE  STAV  pilot  flew  the  different  test  tasks 
in  a  TPS  T-38  so  that  they  could  become  familiar  with  them  and  validate  that  they  were 
operationally  valid  and  safe.  Each  pilot  flew  with  the  TMP  staff  test  pilot  in  the  back 
seat,  so  that  they  fly  and  rate  the  tasks  while  getting  instruction  on  CHR.  The  correction 
altitude  and  magnitude  of  the  offset  were  varied  during  the  lateral  offset  landings  until 
safe  and  operationally  valid  task  parameters  were  decided  upon.  The  same  process  was 
repeated  for  the  vertical  offset  tasks.  The  tasks  were  flown  at  full  flap  and  no  flap 
conditions  to  simulate  the  effects  of  different  pilot  sight  pictures  (the  cockpit  view  a  pilot 
has  when  landing)  during  the  correction  maneuver  and  the  flare.  After  flying  25 
approaches  on  three  sorties,  the  parameters  were  set  at  a  correction  altitude  of  300  feet 
above  ground  level  (AGL)  for  both  offset  tasks  and  offset  magnitudes  of  200  feet  for  the 
lateral  task  and  a  half-dot  (half-degree)  above  glideslope  for  the  vertical  task.  These 
values  were  added  to  the  LAMARS  test  cards  and  lessons  learned  about  pacing  and  test 
conduct  were  explained  to  the  entire  HAVE  STAV  test  team  prior  to  leaving  TPS. 

LAMARS  testing  of  the  version  2  STAV  model  was  conducted  on  6-7  August 
2007  at  Wright-Patterson  Air  Force  Base.  Four  pilots  completed  160  different 
approaches  for  data  during  sixteen  hours  of  testing.  Individual  infonnation  regarding 
these  four  pilots  is  found  in  table  G-2  of  appendix  G.  Three  flight  test  engineers  flew 
approximately  forty  minutes  of  simulation  each  to  prepare  for  TIFS  flight  testing.  Table 
B-l  in  appendix  B  contains  the  entire  test  matrix  used  in  LAMARS  testing.  LAMARS 
was  selected  over  the  ICS  so  that  a  direct  comparison  of  test  data  could  be  made  with  a 
second  round  of  LAMARS  testing  conducted  by  the  Northrop  Grumman  Corporation 


98 


(NGC)  in  November  2006.  The  model  used  in  this  testing  was  also  the  version  2  STAV 
model,  which  now  included  ground  effect  and  gear  modeling,  as  well  as  an  angle  of 
attack  (alpha)  compensation  technique  used  to  reduce  pilot  workload  when  maneuvering. 
The  test  team  wanted  a  high-fidelity  full-motion  simulation  that  replicated  as  closely  as 
possible  the  motions  anticipated  on  TIFS  test  sorties.  A  HUD  was  not  used  during 
LAMARS  testing  because  the  TIFS  cockpit  did  not  have  one.  A  center  inceptor  location 
was  used  instead  of  a  side  stick  in  order  to  better  replicate  the  TIFS  cockpit.  The  main 
objective  was  to  identify  an  optimized  flight  control  system,  feel  system,  or  technique  to 
flight  test  in  the  TIFS  in  addition  to  the  baseline  STAV  model.  Testing  was  conducted  in 
three  phases,  the  first  of  which  investigated  the  flying  qualities  of  the  baseline  STAV 
model  and  compared  the  alpha-command  (angle  of  attack),  gamma-command  (flight 
path),  and  q-command  (pitch  rate)  control  systems.  All  tasks  were  again  designed  to  land 
the  aircraft  1,000  feet  down  the  runway  on  centerline.  The  performance  criteria  were  also 
the  same  as  ICS  testing  except  for  the  desired  and  adequate  sink  rate  criteria,  which  were 
decreased  to  account  for  the  STAV  landing  gear  structural  capabilities.  The  precision 
and  offset  landing  perfonnance  criteria  are  shown  in  table  10. 

Table  10  -  LAMARS  Landing  Criteria 


Precision/  Offset  Landings 

Desired 

Adequate 

Landing  zone 

±25  ft  laterally 
±500  ft  longitudinally 

±50  ft  laterally 
±1000  ft  longitudinally 

Deviation  from  touchdown  airspeed 

±  5  knots 

±10  knots 

Max  bank  angle  below  50  feet 

±  5  degrees 

±  7  degrees 

Max  touchdown  sink  rate 

4  ft/sec 

6  ft/sec 

Deviation  from  runway  heading  at 
touchdown 

±  2  degrees 

±  4  degrees 
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4.3.1  Baseline  STAY  Model 


Results  from  the  first  phase  of  testing  closely  matched  the  results  of 
previous  NGC  LAMARS  control  system  testing.  All  three  HAVE  STAV  pilots  agreed 
that  even  though  it  required  improvement,  the  alpha-command  control  system  should  be 
tested  further  in  TIFS.  The  gamma  controller  was  slightly  less  intuitive  to  the  pilot,  but 
obtained  comparable  results  to  the  alpha  controller  during  low  workload  tasks.  If  no 
large  lateral  corrections  were  required  (due  to  high  crosswinds  or  lateral  offset),  and 
workload  remained  low,  the  gamma  controller  provided  perfonnance  results  comparable 
to  or  slightly  better  than  the  alpha  controller.  However,  in  cases  where  large  lateral 
corrections  were  required,  the  aircraft  motions  and  control  inputs  were  unnatural  to  the 
pilots.  If  actual  instrument  conditions  were  present,  the  pilots  would  easily  become 
spatially  disoriented.  The  pitch  rate  controller  provided  the  biggest  challenge  for  all  of 
the  pilots  and  was  the  most  disorienting  to  use.  It  was  difficult  to  predict  the  response  of 
the  aircraft  to  a  longitudinal  input,  making  it  hard  to  maintain  the  glideslope  and  flare  the 
aircraft.  Each  pilot  developed  a  technique  for  accomplishing  the  flare  during  the  first 
phase,  after  which  the  pilots  decided  on  a  standardized  flare  technique  that  involved 
altitude  calls  by  the  test  conductor  at  AGL  altitudes  of  100,  50,  and  20  feet  and  a  timed 
power  reduction  when  passing  through  20  feet.  At  the  end  of  this  first  phase  of  testing, 
the  team  collectively  decided  to  conduct  all  further  testing  and  control  system 
modifications  with  the  alpha-command  control  system. 

During  the  first  phase  of  testing,  all  pilots  noted  that  the  flare  was  the  most 
difficult  part  of  a  landing  task.  Handling  qualities  during  the  approach  (above  300  feet 
AGL)  were  not  problematic.  In  fact,  pilots  commented  that  maintaining  the  appropriate 
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glideslope  and  alignment  with  the  runway  were  not  challenging,  and  that  the  HQ  should 
be  considered  satisfactory.  However,  once  close  to  the  ground  (below  300  feet  AGL),  the 
longitudinal  inputs  required  to  maneuver  and  flare  the  aircraft  were  difficult  to  control. 
The  flare  typically  required  a  tradeoff  between  satisfying  either  the  landing  distance  or 
the  sink  rate  evaluation  criteria.  When  the  pilot  focused  on  achieving  the  desired  sink 
rate  criterion,  the  typical  result  was  a  landing  distance  of  1500  to  2000  feet  long  of  the 
desired  touchdown  point.  When  the  pilot  focused  on  meeting  the  desired  landing 
distance  criterion,  the  typical  result  was  a  hard  touchdown  with  sink  rate  between  six  and 
ten  feet  per  second.  The  first  phase  of  testing  began  with  the  first  pilot  flying  approaches 
at  175  and  195  knots.  The  second  pilot  flew  at  185  and  195  knots,  and  the  third  pilot  at 
175  and  185  knots.  The  first  pilot  flew  again  at  185  knots,  and  agreed  with  the  other 
pilots  that  185  knots  was  the  best  approach  speed  for  STAV.  This  approach  speed  was 
then  used  in  all  subsequent  testing. 

Before  any  test  runs  were  completed,  a  flying  qualities  check  was  made  on  each 
control  system.  This  check  was  accomplished  via  a  series  of  pilot  inputs  that  included 
steps  and  doublets.  The  aircraft  characteristics,  including  short  period  frequency  and 
damping  and  time  delay,  were  measured  and  compared  to  the  baseline  model 
characteristics.  The  comparison  was  made  by  both  the  HAVE  STAV  test  team  and  an 
NGC  engineer  in  charge  of  STAV  flight  controls.  The  flying  qualities  of  the  STAV  as 
implemented  on  LAMARS  were  the  same  as  those  exhibited  by  the  baseline  STAV 
model  during  previous  testing. 
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4.3.2  Model  Optimization 

During  the  second  phase  of  testing,  the  longitudinal  inceptor  force 
gradient  was  increased  just  prior  to  entering  the  flare.  This  was  done  to  limit  the 
undesired  pitching  motions  and  pilot  tendency  to  over-control  during  the  flare.  Using  the 
procedures  outlined  in  the  LAMARS  testing  section  of  chapter  3,  the  pilots  came  up  with 
optimized  values  for  both  the  force  gradient  and  the  altitude  of  the  gradient  change.  The 
optimal  gradient  was  detennined  to  be  five  times  the  baseline  gradient,  or  approximately 
13.5  pounds  of  force  per  inch  of  inceptor  deflection.  This  gradient  was  a  compromise 
between  the  two  lighter  test  pilots  who  preferred  lighter  inceptor  forces  (four  times  the 
baseline  gradient)  and  the  heavy  test  pilot  who  favored  heavier  inceptor  forces  (seven 
times  the  baseline  gradient).  The  selected  gradient  reduced  the  tendency  to  over-control 
during  the  flare,  and  increased  the  pilot’s  ability  to  make  an  acceptable  landing  even 
when  initially  off  parameters  (i.e.  steep  flight  path  angle  or  high  airspeed).  The  optimal 
height  above  ground  for  the  gradient  change  was  100  feet  AGL.  Below  100  feet  AGL, 
the  gradient  change  had  a  negative  impact  on  the  flare.  Pilots  pulled  aft  on  the  inceptor  to 
begin  the  flare,  and  during  this  pull  the  force  gradient  suddenly  increased,  which  resulted 
in  an  undesirable  increase  in  workload.  Above  100  feet  AGL,  the  gradient  change 
interfered  with  pilot’s  inputs  during  a  lateral  or  vertical  correction,  and  caused  an  increase 
in  workload. 

After  the  increased  longitudinal  inceptor  gradient  testing  finished,  the  effects  of 
spoiler  retraction  during  the  flare  were  investigated.  As  in  both  the  ICS  and  previous 
NGC  LAMARS  testing,  the  spoilers  were  initially  set  to  a  30%  bias  in  order  to  provide 
better  speed  stability  and  control.  The  spoiler  retraction  minimized  the  throttle  change 
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required  to  maintain  airspeed  during  the  flare.  This  led  to  a  more  natural  pitching 
moment  during  the  flare,  and  reduced  the  landing  gear  sink  rate  generated  by  an  aft  pull 
on  the  inceptor.  Pilots  noted  that  the  aircraft  response  to  inceptor  inputs  during  the  flare 
was  more  predictable  when  accompanied  by  the  spoiler  retraction.  An  automatic  spoiler 
retraction  height  of  30  feet  AGL  was  decided  upon  by  the  pilots  as  optimal.  The  altitude 
of  the  retraction  depended  heavily  on  a  pilot’s  flare  technique.  If  the  pilot  attempted  to 
approach  the  landing  zone  with  a  higher  than  normal  airspeed  and  slow  down  during  the 
flare,  then  the  spoilers  would  completely  retract  well  before  touchdown.  If  the  pilot 
attempted  to  approach  the  landing  zone  with  slower  than  normal  speed  and  attempt  to 
make  a  spot  landing,  then  touchdown  would  occur  prior  to  complete  spoiler  retraction. 
The  optimal  altitude  selected  allowed  for  complete  spoiler  retraction  just  as  a  nominal 
touchdown  occurred.  If  touchdown  did  not  occur  within  a  few  seconds  after  complete 
spoiler  retraction,  then  the  aircraft  would  tend  to  “float”  down  the  runway  in  ground 
effect.  This  floating  tendency  sometimes  caused  the  aft  part  of  the  aircraft  to  strike  the 
runway  due  to  dangerously  low  airspeeds  or  high  attitudes.  The  inceptor  force  gradients 
of  both  the  baseline  and  LAMARS  optimized  STAV  control  systems  are  shown  in  table 
11. 


Table  11  -  STAV  Control  Systems 


Control 

System 

Breakout 

Forces 

(Pounds) 

Friction 

Forces 

(Pounds) 

Force  Gradient 
(Pounds/Inch) 

Longitudinal 
Travel  (Inches) 

Alternate  ! 

Control 
Technique 

Baseline 

1 

1 

2.6 

3.2  forward  /  4.2 
aft 

N/A 

LAMARS 

Optimized 

1 

1 

13.5  @  100’  AGL 

3.2  forward  /  4.2 
aft 

Spoilers  retracted 
@  30  ft  AGL 
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4.3.3  Baseline/  LAMARS  Optimized  Model  Comparison  (LAMARS) 

The  flare  HQ  showed  improvement  when  coupling  the  spoiler  retraction 
with  the  increased  longitudinal  inceptor  force  gradient.  When  using  the  optimized 
control  system,  the  handling  qualities  were  regularly  acceptable  or  better  during  the 
landing  tasks  and  were  usually  only  unacceptable  during  high  crosswind  or  lateral  offset 
landing  tasks.  These  results  were  an  improvement  over  the  normally  unacceptable 
baseline  STAV  model  HQ.  Tables  12  and  13  show  the  Cooper-Harper  ratings  for  the 
baseline  and  optimized  systems,  as  well  as  the  performance  achieved  for  both  systems. 
While  the  optimized  system  still  had  a  good  portion  of  inadequate  landings  and  therefore 
unacceptable  HQ,  it  displayed  a  marked  improvement  over  the  baseline  STAV  model. 


Table  12  -  LAMARS  Baseline  vs.  Optimized  CHR 


CHR 

3 

4 

5 

6 

7 

8 

Baseline 

0 

1 

13 

1 

22 

3 

Optimized 

4 

1 

6 

1 

8 

0 

Table  13  -  LAMARS  Baseline  vs.  Optimized  Performance  Achieved 


Desired  (Total  %) 

Adequate  (Total  %) 

Inadequate  (Total  %) 

Baseline 

1  (2.5) 

14(35) 

25  (62.5) 

Optimized 

5(25) 

7(35) 

8  (40) 

The  percentage  of  inadequate  perfonnance  landings  decreased  by  36%  from  the 
baseline,  while  the  percentage  of  landings  with  desired  performance  increased  by  a  factor 
of  ten.  Also  of  note  was  the  near  lack  of  CHR  of  6,  defined  as  “adequate  performance 
requires  extensive  pilot  compensation.”  Pilots  were  generally  not  working  hard  enough 
to  give  a  CHR  of  6.  This  was  due  in  large  part  to  the  lack  of  perceived  sink  rate  by  the 
pilots.  The  pilots  would  think  that  they  were  about  to  make  a  desired  or  adequate 
landing,  but  after  touchdown  would  realize  that  the  sink  rate  was  too  high.  This  nearly 
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imperceptible  sink  rate  prevented  the  pilots  from  working  harder  (extensive  pilot 
compensation)  to  achieve  adequate  landing  criteria,  and  therefore  they  either  gave  a  CHR 
of  5  (adequate  performance  required  considerable  pilot  compensation)  or  a  CHR  of  7 
(adequate  performance  not  attainable  with  maximum  tolerable  pilot  compensation).  The 
CHR  of  7  were  always  based  off  of  inadequate  performance,  not  workload.  The  three 
CHR  of  8  were  assigned  because  the  pilot  encountered  some  undesirable  pitching 
motions  and  considered  them  an  incipient  Pilot  In-the-Loop  Oscillation  (PIO).  These 
motions  were  not  encountered  with  the  optimized  control  system. 

After  the  three  test  team  pilots  had  flown  both  the  baseline  and  optimized 
systems,  the  TPS  staff  pilot  flew  both  systems.  After  analyzing  the  performance 
achieved  and  workload  required,  the  pilot  agreed  with  the  test  team  that  the  handling 
qualities  of  the  optimized  system  were  indeed  better.  The  predictability  and  repeatability 
of  the  optimized  system  in  the  flare,  while  still  not  acceptable,  were  a  marked 
improvement  over  the  baseline. 

When  flying  the  baseline  model,  the  pilot  would  approach  the  landing  and  begin 
to  flare  the  aircraft.  Instead  of  arresting  the  sink  rate,  the  vertical  velocity  would  increase 
and  the  pilot  would  either  impact  the  ground  at  a  high  sink  rate  or  over-control  and  cause 
the  aircraft  to  balloon.  The  optimized  system  showed  no  tendency  to  increase  in  sink  rate 
as  the  inceptor  was  pulled  aft.  The  pilot  would  approach  the  landing,  pull  aft  on  the 
inceptor  to  begin  the  flare,  and  the  sink  rate  would  gradually  decrease  until  touchdown. 
The  sink  rates  encountered  during  a  normal  approach  and  landing  for  both  the  baseline 
and  optimized  control  systems  are  shown  in  figure  40.  This  plot  shows  the  vertical 
velocity  of  the  aircraft  (sink  rate)  in  the  moments  prior  to  touchdown,  not  the  aircraft 
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flight  path.  The  plot  therefore  depicts  a  hard  landing  with  the  baseline  system,  not  a 


balloon. 


Figure  40  -  Sink  Rate  of  Baseline  vs.  Optimized  Systems 
Another  method  was  created  to  determine  differences  between  the  baseline  and 


optimized  systems  after  TIFS  testing  was  completed.  This  method  was  then  applied  to 
the  LAMARS  test  data.  The  inceptor  velocity  was  measured  as  a  function  of  time,  and 
used  as  a  metric  for  pilot  aggressiveness.  The  percentage  of  time  that  the  pilot  was 
moving  the  inceptor  over  a  given  period  was  measured,  and  used  as  a  metric  for  duty 
factor.  These  two  metrics  were  then  plotted  against  one  another  to  determine  if 
aggressiveness  and  duty  factor  differed  between  the  systems  and/or  influenced  pilot 
opinion  on  performance  and  predictability.  Figure  41  depicts  pilot  aggressiveness  and 
duty  factor  for  both  the  baseline  and  optimized  systems.  This  figure  quantifies  the 
physical  workload  as  a  two-dimensional  combination  of  aggressiveness  and  duty  cycle 
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that  serves  as  a  time-domain  analogous  representation  of  the  frequency-domain  concept 
of  “frequency  content.”  Large,  abrupt,  and  frequent  inceptor  motions  are  plotted  in  the 
upper  right  corner  and  are  analogous  to  “high  pilot  gain.”  Conversely,  small,  smooth, 
infrequent  inceptor  motions  are  plotted  in  the  lower  left  comer  and  correspond  to  “low 
pilot  gain.” 


Pilot  Workload  Measured  as  Aggressiveness  vs.  Duty  Factor 
LAMARS  Results 
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Figure  41  -  LAMARS  Pilot  Aggressiveness  vs.  Duty  Factor 
This  analysis  showed  no  significant  differences  between  the  baseline  and 
optimized  systems.  Even  though  the  optimized  system  resulted  in  better  HQ  than  the 
baseline,  both  systems  displayed  a  wide  range  of  achieved  performance,  and  showed  a 
certain  lack  of  predictability.  Consequently,  both  systems  varied  widely  in  overall 
aggressiveness  and  duty  factor.  Although  the  data  were  somewhat  spread,  it  did  portray  a 
general  relationship  between  pilot  aggressiveness  and  duty  factor  for  both  systems.  As 
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the  duty  factor  decreased  the  aggressiveness  tended  to  decrease,  and  as  the  duty  factor 
increased  there  was  a  corresponding  increase  in  aggressiveness. 

After  the  optimized  system  had  been  completely  developed,  the  baseline  system 
was  retested  to  ensure  that  improved  handling  qualities  were  not  attributed  to  practice 
alone.  The  same  tendencies  to  over-control  during  the  flare  were  observed  when  the 
baseline  STAV  model  was  retested.  Task  performance  in  the  flare  was  again 
unpredictable,  and  resulted  in  almost  the  same  number  of  adequate  and  inadequate 
landings.  Table  14  shows  the  original  baseline  performance  achieved  on  the  first  day  of 
testing  compared  to  the  final  baseline  performance  achieved  on  the  second  day  of  testing. 


Table  14  -  LAMARS  Baseline  Performance  Achieved 


Desired  (Total  %) 

Adequate  (Total  %) 

Inadequate  (Total  %) 

Day  1  Baseline 

1  (4.5) 

7(31.8) 

14  (63.6) 

Day  2  Baseline 

0(0) 

7  (38.9) 

11  (61.1) 

After  analyzing  the  LAMARS  results,  some  areas  for  improvement  and  possible 
underlying  impacts  on  testing  were  proposed.  The  selection  of  the  185  knot  approach 
allowed  the  pilots  time  to  acclimate  to  a  single  approach  speed,  and  reduced  the  number 
of  test  variables.  This  allowed  a  more  direct  comparison  to  be  made  between  the  baseline 
and  optimized  system  in  the  limited  test  time  available.  During  the  landing  tasks,  the 
pilots  noted  that  there  was  a  parameter  (perfonnance)  trade-off  when  attempting  to  make 
a  desired  or  adequate  landing.  Either  the  landing  distance  criterion  or  the  sink  rate 
criterion  could  generally  be  met,  but  not  both.  This  relationship  should  be  investigated 
further  in  TIFS.  Both  the  lateral  offset  task  and  crosswinds  increased  the  pilot  gain  to  an 
appropriate  level  while  remaining  operationally  valid.  The  vertical  offset  task  did  not 
drive  up  the  gain  as  much  as  desired,  and  should  be  left  out  during  TIFS  testing.  The 
lateral  offsets  increased  the  pilot  gain  the  most  during  testing.  The  background  of  each 
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pilot  did  not  have  a  significant  impact  on  the  perceived  HQ  ratings.  The  heavy  pilot 
found  that  the  technique  used  to  flare  heavy  aircraft  was  better  suited  in  the  STAV  than 
the  flare  technique  initially  employed  by  the  fighter  pilots,  and  this  technique  became  the 
accepted  test  technique.  The  preferred  force  gradient  was  the  only  other  pilot-specific 
factor  that  arose  during  testing,  and  this  was  settled  via  compromise  between  the  pilots. 

In  addition  to  yielding  several  interesting  results,  the  LAMARS  testing  also 
provided  some  lessons  learned  for  future  testing.  Better  visuals  in  LAMARS  would 
increase  the  realism  of  the  simulation.  The  displays  were  not  bright  enough  to  pick  up  on 
the  very  subtle  visual  cues  available  to  the  pilots  during  the  flare.  A  blended  inceptor 
gradient  change  would  result  in  a  less  disruptive  impact  on  the  pilot  during  the  approach. 
While  the  pilots  liked  the  higher  gradient  and  the  timing  of  the  change,  they  did  not  like 
how  abrupt  it  was.  The  use  of  altitude  calls  and  a  standardized  power  pull  reduced  pilot 
workload  and  increased  consistency,  and  should  be  employed  during  flight  testing. 
Practice  approaches  allowed  the  pilot  to  become  more  familiar  with  flying  the  tasks,  and 
should  be  used  to  the  maximum  extent  possible.  More  approaches  would  also  allow 
more  thorough  testing  of  each  system.  The  touchdown  zone  criteria  should  be  resized  to 
better  reflect  an  operationally  acceptable  landing  area.  Current  large  bomber  aircraft 
routinely  land  up  to  2000  feet  long  of  there  intended  touchdown  point.  A  crosscheck  of 
the  flying  qualities  of  the  STAV  model  as  implemented  on  TIFS  should  be  made  to  again 
ensure  model  fidelity  during  testing. 

Overall,  the  simulator  had  a  higher  fidelity  than  the  ICS  with  its  visuals  and 
motion,  and  was  a  good  preparation  for  the  flight  tests.  It  allowed  the  team  to  test  the 
baseline  system,  develop  an  optimized  system,  and  refine  the  test  methodology  to  make 
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the  flight  testing  more  efficient.  The  exposure  of  the  pilots  and  engineers  to  the  baseline 
STAV  model  limited  the  potential  for  surprises  during  flight  test.  This  preparation 
proved  to  be  critical  because  of  the  limited  flight  test  time  available  to  conduct  a 
thorough  handling  qualities  evaluation.  Turbulence  was  not  implemented  on  LAMARS, 
and  in  future  testing  this  should  be  looked  at  to  detennine  how  the  model  responds  to 
wind  gusts  or  turbulence  prior  to  flight  testing.  This  lack  of  turbulence  meant  that  the 
pilots  were  largely  out  of  the  loop  prior  to  maneuvering  the  aircraft  through  a  task,  and 
did  not  get  used  to  the  inceptor  feel  until  below  300  feet  AGL.  This  prevented  the  pilots 
from  getting  used  to  the  lighter  inceptor  forces  prior  to  the  gradient  change,  and 
precluded  them  from  perceiving  the  control  hannony  mismatch  (an  unwanted  discord 
between  longitudinal  and  lateral  inceptor  forces)  discovered  during  flight  testing. 
Finally,  the  limited  runs  did  not  allow  a  thorough  exploration  of  the  gamma-command 
controller.  The  benefits  of  this  controller  should  be  further  studied,  especially  during  low 
gain  tasks. 

4.4  TIFS  Flight  Testing 

Flight  testing  of  the  version  2  STAV  model  on  the  TIFS  aircraft  was  conducted 
from  10-13  September  2007  in  the  airborne  traffic  pattern  at  Niagara  Falls  International 
Airport.  A  total  of  six  flights  and  ten  hours  were  flown  during  testing,  as  summarized  in 
table  15.  This  included  sixty-seven  different  approaches  for  data.  A  detailed  synopsis  of 
the  test  points  flown  on  TIFS  is  presented  in  table  D-l  in  appendix  D.  A  summary  of  the 
pilots  who  flew  on  TIFS  is  found  in  table  G-3  in  appendix  G.  The  flight  testing 
objectives  were:  to  detennine  the  powered  approach  handling  qualities  of  the  baseline 
STAV  model,  to  compare  the  LAMARS  optimized  control  system  to  the  baseline  STAV 
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control  system,  and  to  detennine  the  flying  qualities  for  the  TIFS  simulation  of  the  STAV 
flight  control  system.  All  figures  in  this  section  and  appendix  E  are  from  the  HAVE 
STAV  Technical  Information  Memorandum  (Speares,  et  al.,  2007). 

Before  the  first  flight,  the  test  team  conducted  ground  training  on  TIFS  to 
familiarize  the  pilots  with  the  displays,  Variable  Stability  System  (VSS),  and  egress 
procedures  of  the  aircraft.  This  allowed  the  team  to  practice  test  team  procedures  on  the 
ground,  which  preserved  actual  flight  time  for  the  test  tasks.  The  test  team  went  through 
the  process  of  engaging  the  VSS  and  transferring  aircraft  control  from  one  cockpit  to 
another,  which  made  the  flight  testing  more  efficient. 

Table  15  -  Summary  of  Test  Flights 


Flight 

Duration 

Description 

Test  Crew 

1 

2.0 

10  Sep  07  141 0L  /  TIFS  flight  2498 

Speares,  Neff,  Porter 

2 

1.0 

1 1  Sep  07  0940L  /  TIFS  flight  2499 

Domsalla,  Cook,  Gray 

3 

2.0 

12  Sep  07  1010L  /  TIFS  flight  2500 

Quashnock,  Porter,  Domsalla 

4 

2.0 

13  Sep  07  0740L  /  TIFS  flight  2501 

Quashnock,  Neff,  Speares 

5 

2.0 

13  Sep  07  1030L  /  TIFS  flight  2502 

Domsalla,  Cook,  Quashnock 

6 

1.0 

13  Sep  07  1510L  /  TIFS  flight  2503 

Speares,  Cook,  Gray 

Each  test  team  pilot  flew  three  test  sorties,  and  each  flight  test  engineer  flew 
either  two  or  three  sorties.  While  the  test  pilot  and  test  conductor  flew  in  the  forward 
evaluation  cockpit,  the  third  member  of  the  test  team  would  fly  in  the  aft  engineering 
compartment,  and  relay  real-time  task  perfonnance  achieved  to  the  forward  evaluation 
cockpit  so  that  CHR  could  be  assigned.  On  the  first  test  flight,  the  rudder  feedback  to  the 
VSS  initially  caused  the  system  to  go  offline.  After  adjusting  this  feedback,  the  VSS 
worked  properly  and  the  flight  continued.  The  VSS  continued  to  work  properly  over  the 
remaining  test  flights,  apart  from  a  small  number  of  nuisance  trips  encountered  as  testing 
progressed.  The  second  test  flight  was  cut  short  by  weather,  which  prevented  testing  the 
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multiple  sorties  planned  for  days  two  and  three  of  the  flight  test  schedule.  Flight  three 
was  the  2500th  TIFS  test  sortie,  and  on  the  fourth  day  of  testing  three  flights  were 
conducted,  allowing  the  test  team  to  complete  the  flight  test  schedule.  The  ability  to 
triple  turn  TIFS  on  a  single  test  day  was  a  testament  to  the  capabilities  of  both  the  aircraft 
and  the  Calspan  personnel  responsible  for  TIFS  operations. 

.  The  TIFS  performance  criteria  were  similar  to  those  used  in  LAMARS  testing. 
The  two  differences  were  the  longitudinal  dimension  changes  of  the  desired  and  adequate 
landing  area,  and  the  removal  of  the  touchdown  airspeed  criteria.  While  an  on-speed 
touchdown  was  desirable,  it  was  not  critical  to  the  landing  HQ  unless  it  deviated  grossly 
from  nominal.  Table  16  lists  the  pilot  perfonnance  criteria  used  during  flight  testing.  In 
addition  to  these  criteria,  touchdown  airspeed  had  to  be  greater  than  165  knots  and 
touchdown  pitch  attitude  had  to  be  less  than  fifteen  degrees.  These  restrictions  were  put 
in  place  to  prevent  a  simulated  runway  strike  with  the  aft  part  of  the  aircraft. 

Table  16  -  TIFS  Performance  Criteria 


Precision  Landing  and  Lateral  Offset  Landing 

Desired 

Adequate 

Landing  zone 

±25  ft  laterally 
+  1000 /-500  ft 
longitudinally 

±50  ft  laterally 
+  1500 /-750  ft 
longitudinally 

Maximum  bank  angle  at  touchdown 

±  5  degrees 

±  7  degrees 

Maximum  touchdown  sink  rate 

4  ft/sec 

6  ft/sec 

Deviation  from  runway  heading  at  touchdown 

±  2  degrees 

±  4  degrees 

4.4.1  Baseline  STAV  Model  (TIFS) 

The  first  test  objective  was  to  detennine  the  powered  approach  handling 
qualities  of  the  baseline  STAV  model.  For  the  tests  completed,  the  baseline  STAV 
handling  qualities  were  predominantly  unacceptable.  A  total  of  thirty-three  approaches 
were  flown  with  the  baseline  feel  system,  with  a  methodical  buildup  in  workload. 
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Normal  approaches  were  flown  first,  followed  by  normal  approaches  with  crosswind. 
Lateral  offsets  were  then  accomplished,  followed  by  lateral  offsets  with  crosswind. 
Cooper-Harper  ratings  given  by  all  pilots  totaled  one  Level  1  rating,  fifteen  Level  2 
ratings,  and  seventeen  Level  3  ratings.  Pilot  In-the-Loop  Oscillation  ratings  were 
assigned  twice,  each  for  non-divergent  oscillatory  motions.  Figure  42  summarizes  the 
CHR  of  both  the  baseline  and  optimized  systems.  Additional  histograms  of  CHR 
assigned  during  testing  are  shown  in  appendix  E,  figures  E-l  through  E-5.  For  all 
approach  types,  the  driving  factor  for  the  unacceptable  handling  qualities  was  inadequate 
task  performance.  For  most  baseline  feel  system  approaches,  the  pilot  workload  and 
compensation  were  both  determined  to  be  acceptable. 


Overall  Comparison  of  Baseline  and  Optimized  Feel  Systems 


Cooper  Harper  Rating 

Figure  42  -  TIFS  Baseline  and  Optimized  CFIR  Summary 
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The  purpose  of  the  different  approach  types  was  to  create  tasks  that  would 
increase  pilot  workload  (while  maintaining  the  same  performance  criteria)  in  order  to 
uncover  key  HQ  characteristics.  The  sequential  workload  buildup  used  by  the  TMP  team 
in  LAMARS  testing  was  again  employed  in  TIFS  using  the  task  order  already  described: 
nonnal  approach,  normal  approach  with  crosswind,  lateral  offset,  and  lateral  offset  with 
crosswind.  In  LAMARS,  the  escalation  in  workload  with  each  task  was  evident  in  both 
pilot  comments  and  performance.  The  actual  workload  buildup  experienced  in  TIFS 
testing  was  different.  As  expected,  the  nonnal  precision  approach  still  required  the 
lowest  workload  and  the  combined  offset  and  crosswind  task  remained  the  highest 
workload,  presumably  due  to  the  complex  combination  of  control  inputs  required. 
However,  the  corrections  and  inceptor  movements  required  to  fly  an  approach  with 
crosswinds  resulted  in  a  higher  pilot  workload  than  the  corrections  and  movements 
required  to  fly  a  lateral  offset  approach. 

While  the  lateral  offset  task  required  a  lower  workload  than  expected,  landing 
performance  achieved  during  these  landings  remained  worse  than  the  performance 
achieved  with  the  nonnal  landings.  Of  nine  lateral  offset  landings,  seven  failed  to  meet 
adequate  criteria  and  none  achieved  desired  criteria.  However,  there  was  no  single 
reason  for  the  performance  inadequacy.  Three  of  the  approaches  failed  to  meet  adequate 
criteria  for  touchdown  distance  (long),  three  for  sink  rate,  and  four  for  excessively  high 
pitch  attitude.  Two  of  these  approaches  had  multiple  perfonnance  inadequacies. 

At  nominal  NC-131H  approach  speeds,  the  TIFS  aircraft  had  the  capability  to 
generate  the  effects  of  up  to  a  fifteen  knot  crosswind  or  negate  an  actual  fifteen  knot 
steady  state  crosswind  using  side  force  generators  on  the  wings.  However,  the  high  hinge 
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forces  present  on  the  side  force  generators  at  the  HAVE  STAV  approach  speed  of  185 
knots  meant  that  the  actual  crosswind  capability  was  limited  to  only  a  seven-knot 
generation  or  reduction  of  crosswind.  For  most  of  the  baseline  approaches  (19  of  33), 
conditions  included  light  to  moderate  turbulence  and  variable  crosswinds  both  with  and 
without  gusts.  In  these  conditions,  removing  crosswinds  was  difficult  for  the  TIFS  to 
manage  without  tripping  the  VSS  by  exceeding  control  surface  limits.  As  a  result,  many 
of  the  “zero-crosswind”  approaches  were  flown  without  crosswind  simulation,  which 
meant  flying  in  actual  crosswinds  ranging  from  zero  to  seven  knots.  These  conditions 
were  perceived  by  the  pilots  to  have  a  higher  workload  than  either  the  lateral  offset  or 
crosswind  tasks  themselves.  Unscheduled  and  unpredictable  disturbances  caused  by 
turbulence  or  wind  gusts  required  the  pilots  to  continually  correct  the  aircraft’s  attitude 
all  the  way  to  simulated  touchdown,  which  meant  a  large  increase  in  pilot  workload. 

The  designed  tasks  as  well  as  the  environmental  conditions  increased  pilot  gain  to 
levels  appropriate  for  the  purposes  of  the  flight  testing.  Neither  the  tasks  nor  the 
conditions  were  assessed  to  be  unrealistic  for  an  operational  bomber  mission.  The 
weather  conditions  in  particular  revealed  the  sometimes  subtle  handling  qualities 
characteristics  of  the  model  during  approach  and  landing.  Although  the  crosswind  tasks 
in  TIFS  increased  pilot  gain,  the  turbulence  encountered  during  flight  test  drove  up  the 
pilot  gain  even  more. 

Sink  rate  at  touchdown  was  the  critical  performance  parameter  responsible  for  the 
Level  2  and  Level  3  HQ  ratings  during  the  approach  and  landing  tasks.  Even  though  the 
longitudinal  landing  zone  criteria  were  increased,  there  was  still  a  trade-off  between  the 
landing  zone  and  sink  rate  parameters.  When  the  desired  sink  rate  was  assiduously 
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pursued,  this  most  often  resulted  in  only  an  adequate  or  inadequate  longitudinal 
touchdown  point  (typically  long).  The  pilots  remarked  that  they  lacked  sufficient  cues  to 
estimate  aircraft  sink  rate.  Due  to  the  touchdown  eye  height  of  the  notional  STAV  (and 
the  corresponding  simulated  touchdown  point),  peripheral  vision  did  not  provide  the 
pilots  a  “ground  rush”  cue  to  arrest  the  sink  rate.  The  lack  of  a  HUD  meant  that  all 
instrumented  cues  required  the  pilot  to  be  “heads  down”  during  the  most  critical  part  of 
the  landing,  the  flare.  The  test  conductor  provided  some  sink  rate  feedback  by  calling 
altitude  remaining  until  touchdown  at  100  feet,  50  feet,  and  every  10  feet  thereafter.  This 
allowed  the  pilots’  eyes  to  remain  outside.  While  these  audio  cues  helped  the  pilots,  they 
were  insufficient.  Other  cues  that  involved  more  than  just  current  aircraft  parameters 
were  needed  but  not  available.  Combining  the  current  aircraft  parameters  with  some  sort 
of  predictive  guidance  information  from  a  flight  director  or  predictive  flight  path  marker 
would  increase  the  STAV  flight  predictability,  particularly  during  flare  and  landing.  This 
predictive  guidance  would  provide  the  pilot  infonnation  on  what  the  aircraft  parameters 
would  be  in  the  near  tenn  future  if  no  inputs  were  made  to  the  throttles  or  inceptor.  A 
flight  director  could  guide  pilot  inputs  in  order  to  achieve  desired  landing  performance. 
Neither  a  flight  director  nor  any  types  of  predictive  guidance  were  used  during  testing. 
Without  these  additional  cues,  the  landing  became  a  mechanical  exercise  where  flare 
height  and  power  reduction  were  determined  strictly  by  altitude.  The  aircraft 
characteristics  and  overall  time  delay  made  it  difficult  to  predictably  flare  and  land  the 
aircraft  in  this  manner. 

The  most  objectionable  flight  control  characteristic  during  a  landing  with  the 
baseline  STAV  model  was  pitch  sensitivity.  The  inceptor  force  gradient  was  2.6  pounds 
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per  inch.  Full  aft  inceptor  deflection  was  4.2  inches  and  required  a  force  of  just  10.92 
pounds.  The  light  control  forces  required  during  the  flare  decreased  predictability  and 
increased  pilot  workload.  The  baseline  inceptor  gains  during  approach  and  landing 
resulted  in  objectionable  inceptor  sensitivity  and  increased  duty  cycle  and  aggressiveness. 
Testing  also  revealed  that  there  was  a  time  delay  in  flight  path  response  on  the  order  of 
one  second,  which  reduced  the  predictability  of  pitch  inputs  and  resulted  in  open-loop, 
methodical  pilot  compensation  during  approach  and  landing.  These  techniques  included 
power  reductions  and  flare  initiation  at  specific  altitudes,  and  were  characterized  by  step 
or  impulse  inputs  that  waited  for  the  aircraft  to  respond  between  inputs. 

While  the  primary  portion  of  the  pilot  comments  concerned  HQ  in  the  pitch  axis, 
some  interesting  commentary  involved  lateral-directional  issues.  Turbulence  cause  a  roll 
sensitivity  in  the  aircraft.  Lateral  accelerations  were  noted  simultaneously  with  aircraft 
roll  rates  when  the  pilot  commanded  a  roll,  a  characteristic  that  was  subtle  but  not 
objectionable.  When  a  roll  rate  was  induced  by  outside  disturbances  such  as  turbulence 
or  wind  gusts,  lateral  accelerations  were  more  apparent,  though  still  not  objectionable. 
Other  notable  commentary  involved  the  alpha  compensation  during  turns.  An  upward 
pitching  moment  was  experienced  when  rolling  into  a  turn  and  a  downward  pitching 
moment  when  rolling  out  of  a  turn.  These  moments  required  the  pilot  to  impart  an 
unnatural  push  when  rolling  into  a  turn  and  an  unnatural  pull  when  rolling  out  of  a  turn. 

4.4.2  Baseline/  LAMARS  Optimized  Model  Comparison  (TIFS) 

The  second  test  objective  was  to  compare  the  LAMARS  optimized  control 
system  to  the  baseline  STAV  control  system.  The  optimized  system  was  identical  to  the 
baseline  system  until  100  feet  AGL,  when  the  longitudinal  force  gradient  was  increased 
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to  five  times  the  baseline  value  over  a  one-second  span.  At  30  feet  AGL,  the  spoilers 
were  automatically  retracted  and  the  aircraft  was  landed.  The  properties  of  the  two 
systems  were  previously  detailed  in  table  10.  A  total  of  thirty-four  approaches  were 
flown  with  the  optimized  system,  again  with  a  methodical  buildup  in  workload.  The 
comparison  of  the  LAMARS  optimized  control  system  with  the  baseline  STAV  control 
system  showed  that  the  optimized  system  had  improved  handling  qualities  over  the 
baseline  system.  The  number  of  landings  which  achieved  desired  perfonnance  nearly 
tripled,  while  the  number  of  inadequate  landings  decreased  by  30%.  This  relationship  is 
portrayed  in  the  previous  figure  42  and  in  figures  E-l  through  E-5  in  appendix  E.  While 
there  was  an  increase  in  performance  over  the  baseline  system,  the  optimized  system  still 
had  almost  twice  as  many  unacceptable  landings  as  satisfactory  landings.  These  results 
indicated  that  the  optimized  system,  while  better  than  the  baseline  system,  still  had  major 
deficiencies  requiring  improvement. 

The  TIFS  testing  showed  a  complete  lack  of  CHR  of  6,  defined  as  “adequate 
performance  requires  extensive  pilot  compensation.”  This  phenomenon  was  exhibited 
first  in  LAMARS,  and  surfaced  again  in  flight  testing.  Pilots  were  generally  not  working 
hard  enough  to  give  a  CHR  of  6.  This  was  due  mostly  to  the  lack  of  perceived  sink  rate 
by  the  pilots,  where  they  would  think  that  they  were  about  to  make  a  desired  or  adequate 
landing,  but  after  touchdown  would  realize  that  the  sink  rate  was  too  high.  This 
unpredictable  sink  rate  again  prevented  the  pilots  from  working  harder  (extensive  pilot 
compensation)  to  achieve  adequate  landing  criteria,  and  therefore  they  either  gave  a  CHR 
of  5  (adequate  performance  required  considerable  pilot  compensation)  or  a  CHR  of  7 
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(adequate  performance  not  attainable  with  maximum  tolerable  pilot  compensation).  The 
CHR  of  7  were  once  again  always  based  off  of  inadequate  performance,  not  workload. 

The  comparison  of  the  optimized  system  with  the  baseline  system  was 
accomplished  by  alternating  between  the  baseline  and  optimized  systems  during  each  test 
flight,  as  shown  in  appendix  D.  This  test  methodology  helped  to  control  some  of  the 
different  influences  on  testing,  including:  weather,  turbulence,  pilot  proficiency,  and 
variations  in  procedure  between  flight  test  engineers.  Each  pilot  had  approximately  three 
flight  hours  for  the  comparison.  For  the  first  hour,  each  pilot  began  with  a  buildup  in 
workload  flying  the  baseline  system.  For  the  second  hour,  each  pilot  repeated  the  tasks 
with  the  optimized  system.  For  the  third  hour,  only  zero-crosswind,  straight-in 
approaches  were  flown,  nominally  alternating  between  two  runs  with  the  baseline  system 
and  two  runs  with  the  optimized  system.  Natural  crosswinds  were  flown  if  it  was 
detennined  that  the  TIFS  was  unable  to  reliably  model  crosswinds  at  the  185  knot 
approach  speed. 

The  optimized  system  resulted  in  an  aircraft  that  was  much  less  sensitive  in  pitch, 
and  was  more  capable  of  achieving  a  repeatable  and  predictable  flare,  even  when  entry 
conditions  to  the  flare  were  varied.  The  optimized  system  required  different  flare  timing 
than  the  baseline  system.  All  three  pilots,  on  their  first  approach  with  the  optimized 
system,  flared  high.  This  difference  in  timing  highlighted  the  fact  that  the  entire  STAV 
approach,  regardless  of  the  control  system,  was  very  reliant  on  open-loop  flying 
technique  rather  than  closed-loop  flying  down  to  landing.  After  an  input  was 
commanded,  the  pilot  waited  for  the  aircraft  to  respond  to  see  what  correction  would  be 
required.  The  correction  for  leveling-off  too  high  required  an  unnaturally  strong  push, 
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instead  of  a  simple  relaxation  of  longitudinal  pull.  This  push  was  more  noticeable  with 
the  increased  inceptor  force  of  the  optimized  system  and  correspondingly  increased  the 
workload.  This  increase  in  workload  led  to  at  least  one  landing  that  achieved  desired 
performance  but  was  deemed  to  require  improvement  because  of  the  moderate  pilot 
workload  required.  Even  with  a  sometimes  increased  workload,  pilot  performance 
tended  to  improve  with  experience,  as  shown  by  the  decrease  of  inadequate  landings 
presented  in  figure  43. 
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Figure  43  -  Inadequate  Landings  by  Pilot  and  Sortie 

Weather,  especially  turbulence  and  wind  gusts,  had  a  significant  impact  on  the 
perceived  handling  qualities  of  each  system.  In  smooth  air,  the  optimized  system  was 
more  conducive  to  Level  1  HQ.  The  baseline  system  was  more  sensitive,  requiring 
extensive  compensation  that  led  to  Level  2  landings  even  when  desired  performance  was 
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achieved.  In  turbulence,  the  optimized  system  made  it  easier  to  compensate  for 
glideslope  deviations  during  the  flare.  However,  both  systems  still  required  extensive 
compensation  during  the  entire  approach  in  the  form  of  small,  frequent  inputs.  The 
optimized  system  did  not  display  the  same  sink  rate  reduction  in  the  flare  during  flight 
test  as  it  did  in  LAMARS.  This  was  most  likely  a  result  of  the  nominal  turbulence 
encountered  on  short  final.  In  the  absence  of  gusts,  the  optimized  system  could  still  be 
flown  to  Level  1  landings,  even  in  moderate  turbulence.  The  inceptor  forces  of  the 
baseline  system,  however,  were  so  light  that  moderate  turbulence  would  cause  the  inertia 
of  the  pilot’s  hand  to  move  the  control,  which  added  to  the  already  considerable 
compensation  required. 

Figure  44  shows  the  difference  in  physical  workload  required  by  the  two  systems, 
and  again  quantifies  the  physical  workload  as  a  combination  of  aggressiveness  and  duty 
cycle  that  characterizes  a  frequency-domain  concept  with  a  time-domain  representation. 
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Pilot  Workload  Measured  As  Aggressiveness  Vs.  Duty  Factor 
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Figure  44  -  TIFS  Pilot  Aggressiveness  vs.  Duty  Factor 
Unlike  LAMARS,  where  there  were  no  apparent  significant  differences  between 
the  baseline  and  optimized  systems,  the  TIFS  data  did  portray  differences  between  the 
two  systems  with  regard  to  their  workload.  The  wide  range  of  aggressiveness  for  the 
baseline  system  indicated  a  lack  of  predictability,  as  a  highly  predictable  system  would 
have  required  the  same  aggressiveness  on  each  approach.  On  average,  the  optimized 
system  required  roughly  half  of  the  aggressiveness  and  a  slightly  decreased  duty  cycle 
compared  to  the  baseline  system.  These  quantitative  descriptions  correlated  well  with  the 
pilots’  comments  of  increased  predictability  and  reduced  workload  when  flying  with  the 
optimized  system.  Similar  to  the  LAMARS  testing,  the  data  again  showed  a  general 
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relationship  between  pilot  aggressiveness  and  duty  factor  for  both  systems.  As  the  duty 
factor  decreased  the  aggressiveness  tended  to  decrease,  and  as  the  duty  factor  increased 
there  was  a  corresponding  increase  in  aggressiveness. 

The  differences  between  the  two  systems  were  most  pronounced  in  the  last  fifteen 
feet  above  simulated  touchdown.  The  sensitivity  of  the  baseline  system  prevented 
precise  control  and  sometimes  led  to  mild,  recognized  pitch  PIOs  as  the  distance  to  the 
runway  decreased.  The  increased  inceptor  forces  of  the  optimized  system  allowed  for 
more  predictable  control  and  for  better  perception  and  correction  of  small  changes  in 
pitch  near  touchdown.  The  baseline  system  produced  a  sinking  sensation  at  these  low 
altitudes,  while  the  optimized  system  did  not.  The  PIO  characteristics  for  the  baseline 
system  were  all  rated  “1”  except  for  two  cases.  In  one  instance,  an  overshoot  in  pitch 
correction  at  10  feet  AGL  resulted  in  tight  control  leading  to  pitch  oscillations  that  were 
not  divergent,  and  a  PIO  rating  of  4.  In  another  instance,  turbulence  on  final  approach 
resulted  in  undesirable  pitch  motions  (2-3  cycles)  which  tended  to  occur  but  did  not  affect 
task  performance.  No  PIO  tendencies  were  observed  with  the  optimized  system,  as 
shown  in  figure  E-6  in  appendix  E. 

Table  E-l  in  appendix  E  summarizes  the  perfonnance  for  all  inadequate  landings. 
Many  of  the  baseline  system  landings  failed  to  meet  adequate  perfonnance  for  more  than 
one  criterion,  while  the  optimized  system  had  only  one  landing  with  more  than  one 
criterion  failed.  During  the  optimized  system  landings  the  aft  part  of  the  STAV  never 
had  a  simulated  runway  strike,  likely  because  the  increased  inceptor  force  inhibited  the 
pilot  from  making  rapid  pulls  while  close  to  the  runway.  The  optimized  system 
inadequate  landings  were  often  a  trade-off  between  longitudinal  displacement  and  sink 
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rate,  both  of  which  relied  on  the  longitudinal  inceptor  inputs  during  the  flare.  The  reason 
for  an  inadequate  landing  therefore  depended  heavily  upon  a  pilot’s  flare  technique. 
Pilots  who  attempted  to  maintain  a  certain  flight  path  angle  and  used  open-loop  inputs  to 
correct  one  parameter  at  a  time  generally  perfonned  the  best.  Pilots  who  attempted  to 
round  out  the  flare  and  control  the  sink  rate  and  landing  distance  simultaneously  usually 
could  not  do  so.  Additionally,  pilots  who  attempted  this  second  method  would  tend  to 
float  down  the  runway,  and  it  was  only  during  this  flare  technique  that  the  aft  part  of  the 
STAV  would  have  a  simulated  runway  strike.  The  open-loop  flare  technique  became  the 
preferred  landing  method. 

As  shown  by  the  data,  pilots  preferred  the  higher  inceptor  gradient  of  the 
LAMARS  optimized  control  system  during  the  approach  and  landing  phases,  but  the 
timing  of  the  gradient  shift  was  undesirable.  During  simulator  testing,  the  change  in 
gradient  at  100  feet  AGL  was  not  objectionable  to  the  pilots,  as  very  few  inceptor  inputs 
were  required  above  this  altitude.  Although  the  change  was  too  abrupt,  pilots  did  not 
object  to  the  timing  of  the  change.  However,  during  flight  testing,  turbulence  required 
frequent  pilot  inputs  above  100  feet  AGL.  Pilots  became  accustomed  to  the  required 
inceptor  inputs  above  100  feet  AGL,  and  then  the  gradient  changed,  which  required 
compensation.  The  gradient  change  timing  had  a  negative  impact  on  the  approach  and 
landing  HQ. 

When  testing  in  LAMARS,  pilots  required  very  few  lateral  corrections  below 
100  feet  AGL.  However,  during  flight  test,  turbulence  and  wind  gusts  required  pilots  to 
make  low  altitude  lateral  corrections.  Since  the  force  gradient  was  increased  only  in  the 
longitudinal  direction,  the  lateral  inceptor  movements  remained  overly  sensitive,  and 
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pilots  commented  that  the  control  harmony  was  poor.  In  an  aircraft  with  good  control 
hannony,  the  forces  required  to  make  both  lateral  and  longitudinal  inceptor  inputs  will 
tend  to  match.  If  the  force  required  to  move  the  inceptor  in  one  axis  is  significantly 
different  from  that  required  in  another  axis,  then  the  control  hannony  is  considered  to  be 
poor.  The  poor  control  harmony  present  on  STAV  decreased  roll  control  predictability 
and  led  to  over-controlling  in  the  roll  axis  when  pilots  corrected  for  turbulence  or  wind 
gusts. 

4.4.3  Flying  Qualities  Determination  and  Comparison 

The  final  test  objective  was  to  determine  the  flying  qualities  for  the  TIFS 
simulation  of  the  STAV  flight  control  system.  Several  Programmed  Test  Inputs  (PTI) 
and  semi-open-loop  capture  tasks  were  perfonned  on  downwind  in  order  to  accomplish 
this  objective.  The  PTI  included  pitch  doublets,  steps  and  frequency  sweeps,  roll  steps, 
and  yaw  doublets  and  steps.  Capture  tasks  were  completed  in  pitch,  roll,  and  heading. 
The  baseline  system  was  the  only  system  tested  during  all  flying  qualities  maneuvers,  as 
the  optimized  system  did  not  engage  until  100  feet  AGL.  Figure  45  shows  a  time  history 
of  a  pitch  doublet  and  the  STAV  model  pitch  rate  response.  Figure  46  shows  a  time 
history  of  a  yaw  doublet  and  the  STAV  model  angle  of  sideslip  response. 
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Short  Period  Analysis  Using  Time  Ratio  Method 
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Figure  45  -  Short  Period  Analysis  Using  Time  Ratio  Method 
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Percentage  of  Longitudinal  Stick  Deflection 


Dutch  Roll  Analysis  Using  Time  Ratio  Method 
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Figure  46  -  Dutch  Roll  Analysis  Using  Time  Ratio  Method 
Table  17  shows  the  short  period  damping  ratio  and  natural  frequency  as 
determined  using  the  time  ratio  method  due  to  the  large  damping  ratio.  It  also  shows  the 
Dutch  roll  damping  ratio  and  natural  frequency,  which  were  again  determined  using  the 
time  ratio  method  (Yechout,  2003)  due  to  the  large  damping  ratio.  Both  the  short  period 
and  Dutch  roll  damping  ratios  and  natural  frequencies  were  within  the  range  of  values 
considered  satisfactory  by  MIL-STD  1797B.  This  information  drove  the  test  team 
investigation  of  other  reasons  for  the  poor  STAY  handling  qualities. 
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Percentage  of  Rudder  Pedal  Deflection 


Table  17  -  Damping  Ratio  and  Natural  Frequency  for  TIFS/STAV 


Mode 

Damping  Ratio 

Natural  Frequency 

Short  Period 

0.78 

2.12  radians/  sec 

Dutch  Roll 

0.80 

1.11  radians/  sec 

Figure  47  shows  a  time  history  of  a  step  PTI  and  the  STAV  model  flight  path 
angle  response.  Initially,  pitch  steps  were  two  seconds  in  duration  before  the  pilot 
recovered.  The  pitch  step  duration  was  then  extended  to  five  seconds  to  account  for  the 
low  frequency  of  the  short  period.  The  initial  flight  path  response  was  a  small  amplitude 
response  in  the  opposite  sense  of  the  commanded  input  (a  non-minimum  phase  response). 
After  a  delay  of  almost  a  second,  the  response  began  to  more  appropriately  follow  both 
the  commanded  sense  and  amplitude  of  the  input.  This  time  delay  in  flight  path  response 
contributed  to  the  approach  and  landing  unpredictability  and  led  to  the  open-loop 
commands  required  for  adequate  landing  perfonnance. 
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Flight  Path  Response  to  Step  Input 
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Figure  47  -  Flight  Path  Response  to  Step  Input 

During  the  capture  tasks,  the  pitch  and  roll  performance  appeared  responsive  for 
an  aircraft  the  size  of  the  STAV.  The  yaw  response  was  slower  than  both  pitch  and  roll, 
and  was  accompanied  by  a  “heaving”  feeling.  Pitch  captures  typically  had  2-3 
overshoots,  and  the  final  attitude  was  difficult  to  predict,  given  the  initial  lag  in  flight 
path  response.  This  was  especially  evident  with  large  pitch  commands.  The  pitch  capture 
results  were  consistent  with  the  flight  path  lag  and  baseline  system  inceptor  sensitivity 
that  adversely  affected  the  approach  and  landing  HQ. 

Rolling  into  a  bank  required  approximately  five  pounds  of  forward  inceptor  force 
to  maintain  level  flight,  and  rolling  out  required  a  five  pound  pull.  Roll  “ratcheting”  at 


129 


Percentage  of  Longitudinal  Stick  Deflection 


bank  angles  greater  than  twenty  degrees  caused  small  lateral  and  vertical  heaving 
motions.  Fifteen  degree  offset  heading  captures  at  bank  angles  of  15-20  degrees  resulted 
in  heading  overshoots  of  about  three  degrees  initially  and  two  degrees  after  returning  to 
wings  level  flight.  The  roll  and  heading  behaviors  were  likely  the  result  of  a  STAV  flight 
control  system  alpha  compensation  feature  that  fed  in  angle  of  attack  with  roll  to  assist  in 
aircraft  maneuvering.  Even  with  the  sometimes  undesirable  motions,  the  roll  and  yaw 
capture  tasks  correlated  well  with  the  approach  and  landing  handling  qualities. 

Overall,  the  TIFS  followed  the  STAV  model  extremely  well.  Figures  48  and  49 
show  the  STAV  model  response  in  pitch  in  both  smooth  and  turbulent  air,  respectively. 
Accurate  model-following  was  seen  by  both  the  similarity  in  shape  and  the  magnitude  of 
the  peaks  The  model  following  displayed  decreased  accuracy  in  turbulent  air,  but  this 
was  due  mostly  to  the  engine  response  and  spool-up  time  of  the  TIFS,  and  not  the  flight 
control  system.  Even  though  the  accuracy  degraded  in  turbulent  air,  the  model  following 
remained  acceptable.  The  accurate  model-following  illustrated  that  the  STAV  handling 
qualities  could  be  determined  using  the  TIFS. 
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Model  Following  of  Pitch  Angle  in  Smooth  Air 


Figure  48  -  Model  Following  of  Pitch  Angle  in  Smooth  Air 
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Pitch  Model  Following  in  Turbulent  Air 
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Figure  49  -  Model  Following  of  Pitch  Angle  in  Turbulent  Air 

After  analyzing  the  flight  test  results,  some  areas  for  improvement  and  possible 
underlying  impacts  on  testing  were  proposed.  The  version  2  STAV  model  had  better  HQ 
than  the  first  version,  but  remained  Level  3  and  required  improvement.  The  number  of 
approaches  conducted  during  testing  was  limited.  More  testing  would  allow  pilots  to 
more  clearly  define  the  changes  required  to  improve  the  system. 

Pilot  inputs  at  low  altitude  can  rapidly  become  aggressive  and  large  amplitude  as 
the  workload  or  stress  is  increased.  A  flight  control  system  that  could  limit  or 
compensate  for  unsafe  pilot  inputs  at  low  altitude  would  increase  safety  in  the  approach 
and  landing  environment.  This  limit  could  be  a  dynamic  stop,  where  a  pilot  would  feel  a 
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“soft”  stop  that  indicated  the  normal  safety  limit.  The  pilot  could  go  beyond  this  stop 
using  increased  force  if  dictated  by  safety  of  flight,  but  this  soft  stop  would  provide 
feedback  to  the  pilot  that  the  normal  zone  of  travel  for  safe  flight  had  been  reached.  This 
could  help  prevent  unnecessarily  large  inceptor  inputs  during  the  flare.  While  version  2 
of  the  STAV  model  did  incorporate  both  ground  effect  and  gear  modeling,  it  did  not 
account  for  control  surface  movements.  There  were  multiple  control  surfaces  used  on 
STAV  which  were  capable  of  actuating  at  tremendous  rates.  This  could  have  had  an 
impact  on  the  STAV  flight  control  system  if  the  structural  effects  of  these  motions  were 
fed  back  into  the  flight  control  algorithm.  Further  testing  should  study  a  higher  fidelity 
STAV  model  that  incorporated  both  of  these  changes. 

Although  the  results  from  testing  indicated  that  the  up  and  away  HQ  were 
acceptable,  testing  should  be  conducted  throughout  the  predicted  aircraft  mission 
envelope  to  see  if  any  other  flight  regimes  exhibit  degraded  HQ.  The  STAV  had  an  eye 
height  far  above  the  ground,  but  this  eye  height  was  not  simulated  precisely  in  TIFS.  The 
difference  between  the  nominal  STAV  eye  height  and  the  TIFS  simulated  eye  height  was 
approximately  ten  feet,  depending  on  approach  angle  of  attack.  This  difference  resulted 
from  conducting  low  approaches  to  “touchdown”  at  twenty  feet  AGL,  which  increased 
the  safety  margin  but  limited  the  ground  rush  cues  normally  available  to  the  pilots.  The 
power  reduction  in  TIFS  was  different  than  LAMARS.  In  LAMARS,  landing  required  a 
reduction  all  the  way  to  idle  power.  In  TIFS  the  reduction  required  for  landing  was  much 
smaller,  and  the  landing  power  was  well  above  idle.  This  difference  in  power  reduction 
technique  initially  increased  the  workload,  until  the  pilots  adapted  and  the  workload 
correspondingly  decreased.  As  mentioned  previously  in  this  chapter,  synthetic  vision 
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will  be  a  vital  component  to  the  STAV  when  flying  it  operationally,  and  the  capabilities 
and  employment  of  such  a  system  need  to  be  thoroughly  investigated. 

4.5  Summary 

This  chapter  presented  the  results  and  analysis  of  all  testing  completed  during  this 
thesis.  Each  of  the  three  main  test  sections:  ICS  testing,  LAMARS  testing,  and  TIFS 
testing,  were  shown  in  detail.  The  ICS  results  and  analysis  summarized  the  results  first 
by  aircraft,  then  by  overall  HQ  rating  and  data  precision,  and  finally  by  HQ  rating  and 
data  precision  according  to  pilot  classification.  The  LAMARS  and  TIFS  results  and 
analysis  covered  the  baseline  and  LAMARS  optimized  models,  as  well  as  the  comparison 
between  the  two.  They  included  pilot  performance,  CHR,  and  a  measure  of  pilot 
workload  vs.  aggressiveness.  Each  section  discussed:  if  the  pilot  ratings  differed 
according  to  classification;  ways  to  improve  the  test  results;  and  underlying  issues  that 
hindered  the  tests  or  proved  to  be  poor  assumptions. 


134 


5.0  Conclusions  and  Recommendations 


5.1  Overview 

This  chapter  summarizes  the  thesis  research,  and  includes  both  conclusions  about 
the  test  data  and  recommendations  for  the  future.  The  chapter  is  divided  into  the  three 
main  test  sections:  Infinity  Cube  Simulator  (ICS)  testing,  Large  Amplitude  Multi-mode 
Aerospace  Research  Simulator  (LAMARS)  testing,  and  Total  In-Flight  Simulator  (TIFS) 
testing.  In  each  section  the  conclusions  are  reviewed  first.  These  conclusions  not  only 
summarize  the  data,  but  also  explain  why  the  results  occurred.  The  recommendations  of 
each  section  are  then  summarized,  and  consist  of  a  list  of  things  that  can  be  done  to  refine 
or  expand  the  testing,  as  well  as  possible  areas  to  explore  in  future  research.  The  chapter 
shows  when  the  recommendations  of  one  test  section  were  used  in  another,  as  well  as 
when  they  were  not  followed  due  to  outside  constraints  or  limitations. 

5.2  Infinity  Cube  Simulator  Testing 

Several  conclusions  and  recommendations  were  made  concerning  ICS  testing. 
The  version  1  STAV  (Supersonic  Tailless  Air  Vehicle)  handling  qualities  (HQ) 
experienced  during  ICS  testing  were  considered  Level  2,  and  required  improvement.  The 
model  itself  required  modification  prior  to  conducting  more  approach  and  landing  tests, 
because  it  did  not  include  ground  effect  or  gear  modeling  of  the  STAV.  The  ICS  had  a 
display  brightness  that  was  too  low  and  a  STAV  model  that  could  not  be  trimmed,  two 
factors  that  negatively  impacted  pilot  perfonnance.  Testing  in  ICS  involved  only 
approach  and  landing  tasks,  an  evaluation  of  the  entire  STAV  envelope  would  uncover 
any  other  areas  with  degraded  HQ.  Future  testing  should  also  include  a  synthetic  vision 
evaluation,  since  the  location  of  the  STAV  cockpit  dictates  the  need  for  such  a  system. 
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Any  further  testing  should  be  conducted  in  an  airborne  simulator  with  a  variable  stability 
system  (VSS),  as  this  would  provide  the  highest-fidelity  simulation. 

The  ICS  testing  showed  that  pilot  background  had  an  impact  not  only  on  Cooper- 
Harper  rating  (CHR),  but  also  on  the  learning  rate  and  the  precision  used  to  complete 
flying  tasks.  The  use  of  a  baseline  aircraft  was  vital  to  ensure  that  pilots  not  trained  in 
rating  HQ  were  correctly  using  the  Cooper-Harper  rating  (CHR)  scale,  and  served  as  a 
basis  by  which  the  results  of  a  group  of  non-test  pilots  could  be  compared  to  test  pilot¬ 
generated  historical  data.  However,  test  pilots  should  conduct  formal  HQ  evaluations, 
because  non-test  pilots  are  less  likely  to  discern  the  reasons  behind  poor  HQ  or  identify 
proper  methods  for  improvement.  They  are  more  apt  to  blame  deficiencies  on  their  own 
piloting  skill  than  on  the  aircraft  itself.  Prior  to  rating  an  aircraft  or  task,  the  pilot  should 
be  able  to  practice  the  task  as  if  they  were  an  operational  pilot.  If  a  pilot  is  to  perform  at 
a  high  level  with  a  limited  workload  during  an  approach  and  landing  HQ  evaluation,  a 
heads  up  display  (HUD)  is  critical,  because  it  allows  the  pilot  to  simultaneously  maintain 
situational  awareness  on  both  the  parameters  of  the  aircraft  and  the  outside  environment. 

Evaluate  an  improved  STAV  model  that  accounted  for  ground  effect  and 
gear  modeling.  (Rl)1  The  lack  of  ground  effect  led  to  a  tendency  to  balloon  in  the  flare. 
A  STAV  without  a  proper  gear  model  will  not  be  able  to  accurately  assess  the  loads 
encountered  during  landing.  An  increased  fidelity  STAV  model  that  accounted  for  both 
ground  effect  and  gear  modeling  would  mitigate  these  deficiencies.  The  version  2  STAV 
model  used  in  subsequent  LAMARS  and  TIFS  testing  incorporated  these  changes. 


1 

Numerals  preceded  by  an  R  within  parentheses  at  the  end  of  a  sentence  correspond  to  the 
recommendation  numbers  of  this  thesis. 
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Use  a  full-motion  simulator  with  better  visuals  and  a  trimmable  model  to 


provide  a  more  realistic  simulation.  (R2)  Full  motion  along  with  brighter  visuals 
would  improve  the  quality  of  the  simulation  by  coupling  the  perceived  visual  response 
with  the  expected  physical  motion.  A  simulator  able  to  operate  a  trimmable  STAV 
model  would  more  accurately  evaluate  pilot  performance  and  CHR.  LAMARS  used  a 
full-motion  trimmable  STAV  model,  but  had  a  poorer  visual  capability  than  the  ICS. 

Conduct  follow-on  STAV  testing  beyond  the  approach  and  landing  phase  of 
flight.  (R3)  Testing  throughout  the  entire  operational  mission  envelope  of  the  aircraft 
would  provide  a  more  thorough  HQ  evaluation,  and  would  uncover  any  other  areas  of 
potentially  degraded  HQ.  Future  testing  should  investigate  a  larger  flight  envelope,  as  it 
was  not  possible  given  the  scope  of  this  thesis. 

Conduct  follow-on  STAV  testing  using  synthetic  vision.  (R4)  Synthetic  vision 
will  be  a  vital  component  to  the  STAV  when  flying  it  operationally.  This  synthetic  vision 
could  range  from  conventional  size  heads  down  displays  to  large,  panoramic  displays  that 
provide  the  pilot  with  a  large  visual  field.  Synthetic  vision  testing  was  limited  by  the 
scope  of  this  thesis. 

Conduct  follow-on  STAV  testing  on  the  VISTA  or  TIFS  variable  stability 
aircraft.  (R5)  The  highest  fidelity  simulations  are  conducted  in  airborne  simulators  that 
have  a  variable  stability  system  (VSS).  A  VSS  would  allow  a  pilot  to  fly  in  one  aircraft 
while  experiencing  and  evaluating  the  handling  qualities  of  another.  The  TIFS  aircraft 
was  used  to  conduct  flight  tests  of  the  STAV. 

Use  only  test  pilots  when  conducting  a  formal  handling  qualities  evaluation. 
(R6)  Pilots  can  and  will  have  different  backgrounds,  but  they  should  all  have  basic  test 
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experience.  Non-test  pilots  are  not  lacking  in  skill,  they  simply  lack  proper  HQ  training. 
Test  pilots  were  used  exclusively  after  ICS  testing. 

Conduct  more  practice  approaches  per  pilot  prior  to  assigning  a  formal 
CHR.  (R7)  This  would  allow  any  learning  effects  to  take  place  prior  to  a  formal  CHR 
evaluation  without  any  artificial  workload  decrease  caused  by  excessive  repetition  of  the 
task.  Subsequent  LAMARS  and  TIFS  testing  incorporated  practice  approaches  into  the 
test  plan. 

Use  a  HUD  when  conducting  an  approach  and  landing  HQ  evaluation.  (R8) 

A  HUD  decreases  workload  because  it  allows  the  pilot  to  simultaneously  perceive  the 
outside  environment  and  the  aircraft  parameters.  A  HUD  does  not  have  to  be  attached  to 
the  aircraft;  it  can  also  be  something  like  a  helmet-mounted  display.  A  HUD  was  not 
used  by  in  subsequent  LAMARS  and  TIFS  testing  due  to  monetary  and  time  constraints. 

5.3  LAMARS  Simulator  Testing 

LAMARS  was  excellent  preparation  for  TIFS  testing,  as  it  provided  a  higher 
fidelity  simulation  than  the  ICS  that  allowed  the  team  to  test  the  baseline  system,  develop 
an  optimized  system,  and  refine  the  test  methodology.  However,  brighter  visuals  in 
LAMARS  with  a  wider  field  of  view  that  allowed  peripheral  cueing  would  improve  the 
fidelity  and  realism  of  the  simulator.  Turbulence  was  not  implemented  on  LAMARS, 
and  in  future  testing  this  should  be  looked  at  to  detennine  the  model  response  to  wind 
gusts  or  turbulence  prior  to  flight  testing.  This  lack  of  turbulence  drove  an  open-loop 
flying  technique  that  prevented  the  pilots  from  perceiving  a  control  harmony  mismatch 
discovered  during  subsequent  flight  testing.  The  flare  and  landing  were  the  most  difficult 
part  of  each  task,  a  result  similar  to  both  ICS  and  previous  NGC  LAMARS  testing.  This 
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was  due  in  large  part  to  the  unpredictable  sink  rate,  which  limited  the  pilot’s  ability  to 
increase  workload  and  achieve  desired  performance. 

The  alpha-command  control  system  selected  as  the  best  option  for  the  baseline 
version  2  STAV  model  still  had  level  2  HQ  at  best  during  approach  and  landing.  The 
benefits  of  the  gamma-command  system  should  be  further  studied,  especially  during  low 
gain  tasks.  The  optimized  system  also  required  improvement,  but  it  was  markedly  better 
than  the  baseline  system,  and  resulted  in  a  more  predictable  flare.  A  blended  inceptor 
gradient  change  would  result  in  a  less  disruptive  impact  on  the  pilot  during  the  approach. 
Both  systems  showed  that  pilots  would  usually  trade-off  between  the  sink  rate  and 
landing  distance  criteria,  and  that  the  pilot  inputs  became  more  aggressive  as  the  time 
spent  moving  the  inceptor  increased.  The  use  of  altitude  calls  and  a  standardized  power 
pull  reduced  pilot  workload  and  increased  consistency,  and  should  be  employed  during 
flight  testing.  As  in  the  ICS  testing,  the  pilot  should  be  able  to  practice  a  task  as  if  they 
were  an  operational  pilot  prior  to  giving  a  CHR.  The  touchdown  zone  criteria  should  be 
resized  to  better  reflect  an  operationally  acceptable  landing  area.  A  crosscheck  of  the 
flying  qualities  of  the  STAV  model  as  implemented  on  TIFS  should  be  made  to  again 
ensure  model  fidelity  during  testing.  Finally,  the  background  of  each  pilot  did  not  have  a 
significant  impact  on  the  perceived  HQ  ratings. 

Improve  the  LAMARS  visuals  by  increasing  the  brightness  and  widening  the 
field  of  view  to  allow  peripheral  cuing.  (R9)  The  LAMARS  displays  were  not  bright  or 
large  enough  to  pick  up  on  the  subtle  visual  and  peripheral  cues  that  would  be  available 
when  landing  in  the  real  world.  While  the  LAMARS  visuals  were  not  improved,  the 
view  from  the  evaluation  cockpit  through  the  TIFS  bubble  canopy  was  excellent. 
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Blend  the  longitudinal  gradient  change  over  a  time  or  altitude  band  to  make 
it  less  perceptible.  (RIO)  While  the  pilots  liked  the  higher  gradient  and  the  timing  of  the 
change,  they  did  not  like  how  abrupt  it  was.  This  abrupt  change  served  as  a  distraction  to 
the  pilots  during  the  flare,  effectively  increasing  the  workload  and  decreasing  the  CHR. 
The  gradient  change  in  TIFS  testing  occurred  over  a  1.5  second  time  span. 

Use  altitude  calls  and  a  set  power  reduction  to  standardize  the  flare 
technique.  (Rll)  Flare  standardization  improved  repeatability  and  decreased  the  pilot 
workload,  particularly  with  no  HUD  available.  It  minimized  the  differences  between 
pilots  during  the  flare.  Flare  standardization  was  used  in  TIFS,  but  had  to  be  modified  to 
account  for  a  different  power  reduction  technique. 

Increase  the  longitudinal  landing  zone  criteria.  (R12)  Current  large  bomber 
aircraft  routinely  land  up  to  2000  feet  long  of  there  intended  touchdown  point.  This 
would  allow  the  pilot  a  larger  area  to  aim  for,  and  would  not  artificially  increase  the  pilot 
gain  by  attempting  a  spot  landing  on  an  area  that  was  too  small.  The  desired  and 
adequate  landing  zones  were  both  increased  for  TIFS  testing,  but  these  too  proved  to  be 
somewhat  restrictive.  Operational  requirements  need  to  drive  the  landing  zone  criteria, 
which  should  be  flexible  enough  to  account  for  adverse  weather  or  emergency  conditions. 

Crosscheck  the  flying  qualities  of  a  model  as  implemented  on  a  simulator 
with  the  flying  qualities  of  the  model  itself.  (R13)  Proper  implementation  of  the  model 
onto  a  simulator  needs  to  be  assured.  If  the  flying  qualities  match,  then  the  results  from 
the  simulations  can  be  assumed  to  be  the  same  as  the  results  one  would  get  when  using 
the  model  itself.  A  series  of  steps,  impulses,  and  simple  capture  tasks  were  performed  on 
TIFS  to  make  sure  the  flying  qualities  matched  the  baseline  STAY  model. 
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5.4  TIFS  Flight  Testing 

The  TIFS  flight  testing  resulted  in  a  number  of  conclusions  and  recommendations. 
As  with  the  LAMARS  testing  conducted  by  the  HAVE  STAV  test  team,  pilot 
background  did  not  have  a  significant  impact  on  the  perceived  HQ.  The  version  2  STAV 
model  had  better  HQ  than  the  first  version,  but  remained  unacceptable  during  approach 
and  landing  tasks.  Pitch  sensitivity  was  the  most  objectionable  flight  control 
characteristic  when  landing  the  baseline  STAV  model.  The  handling  qualities  of  the 
optimized  system  were  better  than  the  baseline  system,  but  required  improvement.  The 
unpredictable  sink  rate  again  limited  the  pilot’s  ability  to  increase  workload  and  achieve 
desired  performance.  Sink  rate  remained  the  critical  perfonnance  parameter  of  both 
systems  during  landing,  and  the  trade-off  between  the  sink  rate  and  landing  distance 
criteria  occurred  once  more.  Both  systems  showed  that  pilot  inputs  became  more 
aggressive  as  the  time  spent  moving  the  inceptor  (duty  cycle)  increased.  On  average,  the 
optimized  system  required  roughly  half  of  the  aggressiveness  and  a  slightly  decreased 
duty  cycle  compared  to  the  baseline  system.  The  increased  inceptor  forces  of  the 
optimized  system  allowed  for  more  predictable  control  and  for  better  perception  and 
correction  of  small  changes  in  pitch  near  touchdown. 

Turbulence,  which  was  not  encountered  during  encountered  during  ICS  or 
LAMARS  testing,  increased  the  pilot  workload  during  flight  test  and  had  an  unexpected 
impact  on  CHR.  Any  decrease  in  pilot  gain  associated  with  the  reduced  7-knot  crosswind 
generation  capability  was  more  than  made  up  for  with  the  response  to  turbulence; 
therefore  this  reduction  in  crosswind  capability  had  no  real  impact  on  testing.  Alpha 
compensation  generated  by  the  flight  control  system  during  turns  caused  moments  that 
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required  the  pilot  to  impart  an  unnatural  push  when  rolling  into  a  turn  and  an  unnatural 
pull  when  rolling  out  of  a  turn.  The  approximately  ten  foot  difference  between  the 
nominal  STAV  eye  height  and  the  TIFS  simulated  eye  height  increased  the  safety  margin 
but  limited  the  ground  rush  cues  normally  available  to  the  pilots.  An  appropriate  HUD 
would  have  improved  the  flight  path  and  sink  rate  awareness  needed  during  the  visual 
portion  of  the  landing.  Combining  the  current  aircraft  parameters  with  some  sort  of 
predictive  guidance  infonnation  from  a  flight  director  or  predictive  flight  path  marker 
would  increase  the  STAV  flight  predictability,  particularly  during  the  flare  and  landing. 
As  shown  in  LAMARS  testing,  a  crosscheck  of  the  flying  qualities  of  a  model  as 
implemented  on  a  simulator  should  always  be  made  to  ensure  model  fidelity  during 
testing.  Again  as  in  both  ICS  and  LAMARS  testing,  the  pilot  should  be  able  to  practice  a 
task  as  if  they  were  an  operational  pilot  prior  to  giving  a  CHR,  while  an  evaluation  of  the 
entire  STAV  envelope  would  uncover  any  other  areas  with  degraded  HQ.  Since  the 
location  of  the  STAV  cockpit  dictates  the  need  for  synthetic  vision,  future  testing  should 
include  such  an  evaluation.  This  evaluation  should  also  involve  both  a  HUD  and 
predictive  guidance,  as  these  three  systems  would  be  heavily  integrated  in  the  STAV. 

The  time  delay  in  flight  path  response  inherent  in  the  flight  control  system 
negatively  affected  aircraft  predictability  in  the  pitch  axis  during  approach  and  landing. 
The  timing  of  the  increase  in  inceptor  force  gradient  was  inappropriate,  and  forced  pilots 
to  dramatically  increase  workload  at  low  altitudes.  The  lateral  inceptor  force  gradient  did 
not  change  when  the  longitudinal  gradient  increased,  and  this  adversely  affected  control 
hannony.  The  results  of  both  systems  showed  that  pilot  inputs  at  low  altitude  can  rapidly 
become  aggressive  and  large  amplitude  as  the  workload  or  stress  is  increased,  causing  a 
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potentially  dangerous  situation.  While  version  2  of  the  STAV  model  did  incorporate  both 
ground  effect  and  gear  modeling,  it  did  not  account  for  the  multiple  control  surface 
movements  present  when  maneuvering,  which  could  have  an  impact  on  the  STAV  flight 
control  system  if  the  structural  effects  of  these  motions  were  fed  back  into  the  flight 
control  algorithm. 

Increase  the  inceptor  force  gradient  for  approach  and  landing.  (R14) 

Baseline  inceptor  gains  were  too  low  during  approach  and  landing,  resulting  in  a  loose 
feel,  objectionable  inceptor  sensitivity,  and  increased  duty  cycle  and  aggressiveness.  Any 
increase  in  inceptor  force  gradient  should  include  both  longitudinal  and  lateral  changes  in 
order  to  preserve  control  hannony. 

Test  a  model’s  response  to  turbulence  prior  to  flight  testing.  (R15) 

Turbulence  can  have  an  impact  both  on  the  pilot  workload  and  on  the  model  following 
capabilities  of  a  simulation.  Testing  in  turbulence  would  better  simulate  real  world 
conditions  and  an  aircraft’s  response  to  those  conditions.  Investigating  turbulence  prior 
to  flight  testing  would  save  both  time  and  money 

Reduce  the  amount  of  alpha  compensation  generated  during  turns.  (R16) 
When  attempting  to  compensate  for  the  increased  angle  of  attack  in  a  turn  by  generating  a 
pitching  moment  to  aid  the  pilot,  the  flight  control  system  overcompensated  with  too  high 
a  moment  that  forced  unnatural  pilot  inputs. 

Implement  a  HUD  on  the  STAV.  (R17)  A  HUD  would  have  provided 
simultaneous  situational  awareness  of  both  the  aircraft  parameters  and  the  outside 
environment.  In  addition,  previous  NGC  LAMARS  testing  indicated  that  powered 
approach  and  landing  handling  qualities  were  improved  when  using  a  HUD. 
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Implement  predictive  guidance  on  the  STAV.  (R18)  This  predictive  guidance 
would  provide  the  pilot  information  on  what  the  aircraft  parameters  would  be  in  the  near 
tenn  future  if  no  inputs  were  made  to  the  throttles  or  inceptor,  and  could  guide  pilot 
inputs  in  order  to  achieve  desired  landing  performance.  Predictive  guidance  could  be 
employed  on  a  HUD  using  synthetic  vision  to  aid  the  pilot  not  only  during  approach  and 
landing,  but  also  during  other  more  mission-critical  tasks. 

Reduce  the  time  delay  in  flight  path  response.  (R19)  Time  delay  in  flight  path 
response,  on  the  order  of  one  second,  reduced  predictability  of  pitch  inputs  and  resulted 
in  open-loop,  methodical  pilot  compensation  during  approach  and  landing.  A  reduction 
in  the  flight  path  response  time  delay  would  result  in  improved  HQ. 

Provide  more  time  for  the  pilot  to  acclimate  to  inceptor  force  gradient 
changes  prior  to  touchdown.  (R20)  Pilots  commented  that  it  would  have  been  desirable 
to  have  the  same  inceptor  force  gradient  for  the  entire  final  approach.  The  timing  of  such 
an  inceptor  force  gradient  change  could  be  similar  to  another  highly-augmented  military 
aircraft,  the  F-16. 

Change  the  lateral  inceptor  force  gradient  to  preserve  control  harmony. 
(R21)  With  no  increase  in  force  gradient  the  lateral  inceptor  movements  remained  overly 
sensitive,  resulting  in  a  poor  control  harmony  that  decreased  roll  control  predictability 
and  led  to  over-controlling  in  the  roll  axis  when  pilots  corrected  for  turbulence  or  wind 
gusts. 

Implement  automatic  approach  and  landing  safety  compensation  on  the 
STAV.  (R22)  A  flight  control  system  that  could  limit  or  compensate  for  unsafe  pilot 
inputs  at  low  altitude  would  increase  safety  in  the  approach  and  landing  environment. 
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Implement  a  higher-fidelity  STAV  model  that  accounted  for  the  impact  of 
control  surface  movements  on  the  flight  control  system.  (R23)  There  were  multiple 
control  surfaces  capable  of  actuating  at  tremendous  rates  used  on  STAV,  which  could 
potentially  feed  back  into  the  flight  control  algorithm.  Further  testing  should  study  a 
higher  fidelity  STAV  model  that  accounted  for  these  movements. 
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Vita 
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completed  the  course  and  was  assigned  to  fly  the  F-15E  at  Seymour  Johnson  AFB,  North 
Carolina,  where  he  served  as  a  combat  fighter  pilot  of  the  336  FS  Rocketeers.  From 
January  to  May  of  2003  he  flew  46  combat  missions  in  Operations  Southern  Watch  and 
Iraqi  Freedom,  and  was  awarded  two  Distinguished  Flying  Crosses  for  his  combat 
actions.  While  at  Seymour  Johnson,  Captain  Speares  was  selected  to  the  Air  Force 
Institute  of  Technology/Test  Pilot  School  (AFIT/TPS)  program.  He  participated  in  initial 
STAV  simulator  testing  conducted  by  the  NGC  in  LAMARS  in  the  spring  of  2006.  He 
was  a  top  third  graduate  of  USAF  Test  Pilot  School  Class  07A  and  is  currently  assigned 
to  the  445th  Test  Operations  Squadron  at  Edwards  AFB.  He  has  over  1200  hours  in  over 
thirty  different  aircraft,  including  850  in  the  F-15E. 
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Appendix  A  -  Additional  Infinity  Cube  Simulator  Results 


Table  A-l  -  ICS  CHR  Summary 


Pilot  Type 

T38  LO 

T38  LA 

175  LO 

175  LA 

195  LO 

195  LA 

Overall  Mean 

3.07 

2.94 

5.22 

5.09 

Overall  a 

1.30 

1.19 

1.73 

1.85 

USAF  Mean 

5.68 

5.31 

USAFct 

1.78 

1.92 

Navy  Mean 

EH 

Navy  a 

MEM 

Fighter  Mean 

3.30 

2.89 

5.33 

5.25 

Fighter  a 

0.97 

1.19 

1.68 

2.05 

Heavy  Mean 

EH 

Heavy  a 

mm 

Non-test  Mean 

3.03 

3.81 

Non-test  a 

1.33 

Test  Mean 

3.25 

3.63 

5.96 

5.92 

Test  a 

1.18 

1.63 

0.98 

1.20 
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Appendix  B  -  LAMARS  Test  Matrix 


Table  B-l  -  LAMARS  Test  Matrix 


Key  of  Abbreviations  in  Modeling  and  Simulation  Matrix 

Pilot 

1  Speares 

2  Domsalla 

3  Quashnock 

4  Gray 

Task 

N  Normal 

L  Lateral  Offset 

V  Vertical  Offset 

Feel  System 

B  Baseline 

IS  Inc  Inceptor  Force 

SP  Spoiler  Reset 

IS/SP  Combined 

Control  Type 

A  Alpha 

G  Gamma 

P  Pitch  Rate 

Crosswind 

0  Zero 

M  Max 

Airspeed 

L  175 

H  195 

Hour  # 

Pilot 

Run  # 

Control 

Type 

Feel 

System 

Airspeed 

Task 

Crosswind 

1 

1 

1 

A 

B 

L 

N 

O 

1 

1 

2 

A 

B 

L 

N 

O 

1 

1 

3 

A 

B 

L 

N 

o 

1 

1 

4 

A 

B 

L 

N 

M 

1 

1 

5 

A 

B 

L 

L 

O 

1 

1 

6 

A 

B 

L 

L 

M 

1 

1 

7 

A 

B 

H 

N 

O 

1 

1 

8 

A 

B 

H 

N 

M 

1 

1 

9 

A 

B 

H 

L 

O 

1 

1 

10 

A 

B 

H 

L 

M 

2 

1 

1 

G 

B 

L 

N 

O 

2 

1 

2 

G 

B 

L 

L 

M 

2 

1 

3 

G 

B 

H 

N 

O 

2 

1 

4 

G 

B 

H 

L 

M 

2 

1 

5 

P 

B 

L 

N 

O 

2 

1 

6 

P 

B 

L 

L 

M 

2 

1 

7 

P 

B 

H 

N 

O 

2 

1 

8 

P 

B 

H 

L 

M 

2 

1 

9 

A 

B 

L 

V 

O 

2 

1 

10 

A 

B 

L 

V 

M 

3 

2 

1 

A 

B 

L 

N 

O 

3 

2 

2 

A 

B 

L 

N 

O 

3 

2 

3 

A 

B 

L 

N 

o 

3 

2 

4 

A 

B 

L 

N 

M 

3 

2 

5 

A 

B 

L 

L 

O 

3 

2 

6 

A 

B 

L 

L 

M 

3 

2 

7 

A 

B 

H 

N 

O 

3 

2 

8 

A 

B 

H 

N 

M 

3 

2 

9 

A 

B 

H 

L 

O 

3 

2 

10 

A 

B 

H 

L 

M 
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Hour  # 


Table  B-l  -  LAMARS  Test  Matrix  (Continued) 
Run  #  I  Control  I  Feel  I  Airspeed  \ 


Control 

Type 

Feel 

System 

Airspeed 

Tas 

ik 

A 

IS 

L/H 

N 

A 

IS 

L/H 

N 

A 

IS 

L/H 

L 

A 

IS 

L/H 

L 

A 

SP 

L/H 

N 

A 

SP 

L/H 

N 

A 

SP 

L/H 

L 

A 

SP 

L/H 

L 

A 

SP 

L/H 

N 

A 

SP 

L/H 

L 

A 

B 

L 

N 

A 

SP 

L/H 

N 

A 

SP 

L/H 

N 

A 

SP 

L/H 

L 

A 

SP 

L/H 

L 

A 

IS/SP 

L/H 

N 

A 

IS/SP 

L/H 

N 

A 

IS/SP 

L/H 

L 

A 

IS/SP 

L/H 

L 

G/P 

IS/SP 

L/H 

N 

A 

B 

L 

N 

A 

SP 

L/H 

N 

A 

SP 

L/H 

N 

A 

SP 

L/H 

L 

A 

SP 

L/H 

L 

A 

IS/SP 

L/H 

N 

A 

IS/SP 

L/H 

N 

A 

IS/SP 

L/H 

L 

A 

IS/SP 

L/H 

L 

G/P 

IS/SP 

L/H 

N 

A 

B 

L 

N 

A 

IS/SP 

L/H 

N 

A 

IS/SP 

L/H 

N 

A 

IS/SP 

L/H 

L 

A 

IS/SP 

L/H 

L 

G 

IS/SP 

L/H 

N 

G 

IS/SP 

L/H 

L 

P 

IS/SP 

L/H 

N 

P 

IS/SP 

L/H 

L 

G/P 

IS/SP 

L/H 

N 

A 

B 

L 

N 

A 

B 

L/H 

N 

A 

B 

L/H 

N 

A 

B 

L/H 

L 

A 

B 

L/H 

L 

A 

IS/SP 

L/H 

N 

A 

IS/SP 

L/H 

N 

A 

IS/SP 

L/H 

L 

A 

IS/SP 

L/H 

L 

G/P 

IS/SP 

L/H 

N 

Crosswind 


Table  B-l  -  LAMARS  Test  Matrix  (Continued) 


Hour  # 

Pilot 

Run  # 

Control 

Type 

Feel 

System 

Airspeed 

Task 

Crosswind 

14 

1 

1 

A 

IS/SP 

L 

V 

O 

14 

1 

2 

A 

IS/SP 

L 

V 

M 

14 

1 

3 

A 

IS/SP 

H 

V 

O 

14 

1 

4 

A 

IS/SP 

H 

V 

M 

14 

1 

5 

G 

IS/SP 

L 

V 

O 

14 

1 

6 

G 

IS/SP 

H 

V 

O 

14 

1 

7 

G 

IS/SP 

L/H 

V 

M 

14 

1 

8 

P 

IS/SP 

L 

V 

O 

14 

1 

9 

P 

IS/SP 

H 

V 

O 

14 

1 

10 

P 

IS/SP 

L/H 

V 

M 

15 

2 

1 

A 

IS/SP 

L 

V 

O 

15 

2 

2 

A 

IS/SP 

L 

V 

M 

15 

2 

3 

A 

IS/SP 

H 

V 

O 

15 

2 

4 

A 

IS/SP 

H 

V 

M 

15 

2 

5 

G 

IS/SP 

L 

V 

O 

15 

2 

6 

G 

IS/SP 

H 

V 

O 

15 

2 

7 

G 

IS/SP 

L/H 

V 

M 

15 

2 

8 

P 

IS/SP 

L 

V 

O 

15 

2 

9 

P 

IS/SP 

H 

V 

O 

15 

2 

10 

P 

IS/SP 

L/H 

V 

M 

16 

3 

1 

A 

IS/SP 

L 

V 

O 

16 

3 

2 

A 

IS/SP 

L 

V 

M 

16 

3 

3 

A 

IS/SP 

H 

V 

O 

16 

3 

4 

A 

IS/SP 

H 

V 

M 

16 

3 

5 

G 

IS/SP 

L 

V 

O 

16 

3 

6 

G 

IS/SP 

H 

V 

O 

16 

3 

7 

G 

IS/SP 

L/H 

V 

M 

16 

3 

8 

P 

IS/SP 

L 

V 

O 

16 

3 

9 

P 

IS/SP 

H 

V 

O 

16 

3 

10 

P 

IS/SP 

L/H 

V 

M 

17 

Neff 

1 

A 

B 

L/H 

N 

O 

17 

Neff 

2 

A 

B 

L/H 

L 

O 

17 

Neff 

3 

A 

B 

L/H 

L 

M 

17 

Neff 

4 

A 

IS/SP 

L/H 

N 

O 

17 

Neff 

5 

A 

IS/SP 

L/H 

L 

O 

17 

Neff 

6 

A 

IS/SP 

L/H 

L 

M 

17 

Cook 

1 

A 

B 

L/H 

N 

O 

17 

Cook 

2 

A 

B 

L/H 

L 

O 

17 

Cook 

3 

A 

B 

L/H 

L 

M 

17 

Cook 

4 

A 

IS/SP 

L/H 

N 

O 

18 

Cook 

5 

A 

IS/SP 

L/H 

L 

O 

18 

Cook 

6 

A 

IS/SP 

L/H 

L 

M 

18 

Porter 

1 

A 

B 

L/H 

N 

O 

18 

Porter 

2 

A 

B 

L/H 

L 

O 

18 

Porter 

3 

A 

B 

L/H 

L 

M 

18 

Porter 

4 

A 

IS/SP 

L/H 

N 

O 

18 

Porter 

5 

A 

IS/SP 

L/H 

L 

O 

18 

Porter 

6 

A 

IS/SP 

L/H 

L 

M 
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Appendix  C:  Test  Cards 


Infinity  Cube  Simulator  Test  Cards 

The  following  test  cards  were  used  during  the  Infinity  Cube  Simulator  (ICS) 
testing.  Prior  to  testing,  all  cards  were  briefed  in  detail  to  the  pilots. 


TASK  ID  TASK 

STAV-1  SIM  Precision  Approach  and  Landing 


UNCLASSIFIED 


HAVE  STAV 


FLIGHT  PHASE 

TASK  DESCRIPTION 

PILOT 

DATE 

RUN  NUMBER 

Approach  and  Landing 

Approach  and  Landing 

FIXED  PARAMETERS 

EVALUATION  SEGMENT  1 

LONG  CHR 

LAT/DIR  CHR 

1  Wind:  Calm 

1  GW:  232683  lb  for  STAV  1 

Glideslope  intercept  and  Localizer  track 

Initial  Speed:  220  knots 
Initial  Heading:  On  LOC 
ALT:  1500  ft 

Landing  Speed:  160  knots 


CG  pos  FS:  1044.14  in 
BL:  -0.04  in 
WL:  107.27 


Config:  Gear  down,  30 
deg  spoiler  bias,  mid-fuel 
weight  with  payload 
RWY  surface:  Concrete 


VARIED  PARAMETERS 


Approach  Airspeed: 
195,  175  knots 


TEST  PROCEDURE 


PILOT 

1 .  Establish  aircraft  in  steady  level  flight  at  the  initial  conditions. 

2.  Slow  to  approach  speed  from  220  knots  with  engine  throttles. 

3.  Capture  approach  speed. 

4.  Track  LOC  to  G/S  intercept  and  capture  G/S  (3  deg). 

5.  Continue  approach  to  landing.  Touchdown  target  is  designated  on  runway 
(CAPT  bars). 


TEST  ENGINEER/PILOT  NOT  FLYING 

Record  pilot  comments  and  CHR  for  each  run. 
Record  parameters  at  1000  ft  AGL  and  touchdown. 


Start  Evaluation:  1500  ft,  Approach  speed,  Level 

End  Evaluation:  400  ft  AGL,  Approach  speed,  Descending _ 

EVALUATION  BASIS 

Evaluate  the  ability  to  maneuver  onto  and  track  the  final  approach  path.  Attain  trimmed 
flight  before  the  middle  marker  (approx  0.5  nm  from  the  end  of  the  runway). 

PERFORMANCE  STANDARDS  I  TARGET  |  DESIRED  I  ADEQU 


Deviation  from  Approach  A/S  (KEAS)  0  +/  -  5  +/  - 1 0 

Deviation  from  Glideslope  0  +/  -  0.5  dot  +/  - 1 .0  dot 

Deviation  from  Localizer  0  +/  -  0.5  dot  +/  - 1 .0  dot 

EVALUATION  SEGMENT  2  LONG  CHR  LAT/DIR  CHR 

Precision  Landing 

Start  Evaluation:  400  ft  AGL,  Approach  speed,  Descending 
End  Evaluation:  Touchdown 


EVALUATION  BASIS 

Evaluate  handling  qualities  of  the  airplane  during  landing.  There  should  be  no  tendency 
to  bobble  in  pitch  or  roll  or  for  PIOs. 

PERFORMANCE  STANDARDS  I  TARGET  |  DESIRED  I  ADEQUy 


Landing  Zone  (for  ref.  point  btwn  main  gear)  (ft) 
Maximum  Bank  Angle  Below  50  ft  AGL  (deg) 
Deviation  from  Land  A/S  @  Touchdown  (KEAS) 
Maximum  Touchdown  Sink  Rate  (ft/  sec) 

Deviation  from  RWY  heading  @  Touchdown  (deg) 


TARGET 

DESIRED 

Aim  Point 

50  x  500 

0 

+/  -  5 

0 

+1-5 

<6 

6 

0 

+1-2 

100 x 1000 
+1-7 
+/-  10 
10 
+1-4 


Figure  C-l  -  ICS  Test  Card  1 
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TASK  ID  TASK 

STAV-2  SIM  Lateral  Offset  Landing 


FLIGHT  PHASE  TASK  DESCRIPTION 

Landing  Precision  Landing  from  Lateral  Offset 


UNCLASSIFIED 


HAVE  STAV 


FIXED  PARAMETERS 

Wind:  Calm 

Config:  Gear  down,  30 
deg  spoiler  bias,  mid-fuel 
weight  with  payload 

RWY  surface:  Concrete 

Initial  Position:  8  NM  out 
Initial  Speed:  220  knots 
Initial  Heading:  On  LOC 
ALT:  1500  ft 

Landing  Speed:  160  knots 

Lat  Offset:  +/-  200  ft  offset 
from  LOC 

Cloud  Breakout:  300  ft  AGL 

GW:  232683  lb  for  STAV 

CG  pos:  Same  as  1 

VARIED  PARAMETERS 


Approach  Airspeed: 
195,  175  knots 


TEST  PROCEDURE 


PILOT 

1.  Perform  normal  approach  and  landing  attitude  corrections. 

2.  At  “cloud  breakout”,  visually  correct  course  to  recapture  runway  centerline. 

3.  Continue  to  fly  final  approach  until  touchdown. 

4.  Touchdown  target  is  designated  on  runway. 


EVALUATION  SEGMENT  LONG  CHR  LAT/DI 

Precision  Landing  from  Lateral  Offset 

Start  Evaluation:  600  ft  AGL,  Approach  speed,  Descending 

End  Evaluation:  Touchdown _ 

EVALUATION  BASIS 

Evaluate  the  handling  qualities  in  landing  in  a  high-gain  task.  There  should  be  no 
tendency  to  bobble  in  pitch  or  roll  or  for  PIOs. 


PERFORMANCE  STANDARDS 

TARGET 

DESIRED 

ADEQUATE 

Landing  Zone  (for  ref.  point  btwn  main  gear)  (ft) 

Aim  Point 

50x500 

100x1000 

Maximum  Bank  Angle  Below  50  ft  AGL  (deg) 

0 

+/  -  5 

+1-7 

Deviation  from  Land  A/S  @  Touchdown  (KEAS) 

0 

+1-5 

+/-  10 

Maximum  Touchdown  Sink  Rate  (ft/  sec) 

<6 

6 

10 

Deviation  from  RWY  heading  @  Touchdown  (deg) 

0 

+1-2 

+1-4 

TEST  ENGINEER/PILOT  NOT  FLYING 

Record  pilot  comments  and  CHR  for  each  run. 
Record  parameters  at  touchdown. 


Figure  C-2  -  ICS  Test  Card  2 
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TASK  ID  TASK 

STAV-3  SIM  Vertical  Offset  Landing 


UNCLASSIFIED 


HAVE  STAV 


FLIGHT  PHASE  TASK  DESCRIPTION 

Landing  Precision  Landing  from  Vertical  Offset 


FIXED  PARAMETERS 

Wind:  Calm 

Config:  Gear  down,  30 
deg  spoiler  bias,  mid-fuel 
weight  with  payload 

RWY  surface:  Concrete 

Initial  Position:  8  NM  out 
Initial  Speed:  220  knots 
Initial  Heading:  On  LOC 
ALT:  1500  ft 

Landing  Speed:  160  knots 

Long  Offset:  +  50  ft  vertical 
offset  from  G/S 

Cloud  Breakout:  300  ft  AGL 
GW:  232683  lb  for  STAV 

CG  pos:  Same  as  1 

VARIED  PARAMETERS 

Approach  Airspeed: 

1 95,  1 75  knots 

TEST  PROCEDURE 


PILOT 

1.  Perform  normal  approach  and  landing  attitude  corrections. 

2.  At  “cloud  breakout”,  visually  correct  glideslope  to  make  touchdown  at  target. 
Maintain  desired  vertical  velocity  and  approach  speed. 

3.  Continue  to  fly  final  approach  until  touchdown. 

4.  Touchdown  target  is  designated  on  runway. 


TEST  ENGINEER/PILOT  NOT  FLYING 

Record  pilot  comments  and  CHR  for  each  run. 


PILOT  date  run  number 

EVALUATION  SEGMENT  LONG  CHR  LAT/DIR  CHR 

Precision  Landing  from  Lateral  Offset 

Start  Evaluation:  600  ft  AGL,  Approach  speed,  Descending 

End  Evaluation:  Touchdown _ 

EVALUATION  BASIS 

Evaluate  the  handling  qualities  in  landing  in  a  high-gain  task.  There  should  be  no 
tendency  to  bobble  in  pitch  or  roll  or  for  PIOs. 


PERFORMANCE  STANDARDS 

TARGET 

DESIRED 

ADEQUATE 

Landing  Zone  (for  ref.  point  btwn  main  gear)  (ft) 

Aim  Point 

50x500 

100x1000 

Maximum  Bank  Angle  Below  50  ft  AGL  (deg) 

0 

+1-5 

+1-7 

Deviation  from  Land  A/S  @  T ouchdown  (KEAS) 

0 

+1-5 

+1-  10 

Maximum  Touchdown  Sink  Rate  (ft/  sec) 

<6 

6 

10 

Deviation  from  RWY  heading  @  Touchdown  (deg) 

0 

+1-2 

+1-4 

Record  parameters  at  touchdown. 


UNCLASSIFIED 


Figure  C-3  -  ICS  Test  Card  3 
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LAMARS  Test  Cards 


The  following  test  cards  were  used  during  LAMARS  testing  at  Wright-Patterson 
AFB,  OH. 


TASK  ID  TASK 

STAV-1  CALM  Precision  Landing 


FLIGHT  PHASE  TASK  DESCRIPTION 

Landing  Precision  Landing  from  Normal  Approach 


UNCLASSIFIED 


HAVE  STAV 


FIXED  PARAMETERS 

Wind:  Calm 

Config:  Gear  down,  30 
deg  spoiler  bias,  mid-fuel 
weight  with  payload 

RWY  surface:  Concrete 

Initial  Position:  4  NM  out 
Initial  Speed:  200  knots 
Initial  Heading:  On  LOC 
ALT:  1000  ft  AGL 
Landing  Speed:  Vapp-10 

GW:  232683  lb  for  STAV 

CG  pos  FS:  1044.14  in 

BL:  -0.04  in 

WL:  107.27 

VARIED  PARAMETERS 


TEST  PROCEDURE 


PILOT 

1.  Perform  normal  approach  and  landing  attitude  corrections. 

2.  Continue  to  fly  final  approach  until  flare. 

3.  Touchdown  target  is  designated  on  runway. 


EVALUATION  SEGMENT  - Ld 

Precision  Landing  from  Normal  approach 

Start  Evaluation:  600  ft  AGL,  Approach  speed,  Descending 
End  Evaluation:  Simulated  Touchdown 


EVALUATION  BASIS 

Evaluate  handling  qualities  when  landing  in  a  high-gain  task.  There  should  be  no 


Approach  Airspeed: 

175 

195 

Control  System: 

Alpha 

Gamma 

Pitch  Rate 

PERFORMANCE  STANDARDS 

TARGET 

DESIRED 

ADEQUATE 

Landing  Zone  (for  ref.  point  btwn  main  gear)  (ft) 

Aim  Point 

50  x  500 

100 x 1000 

Maximum  Bank  Angle  Below  50  ft  AGL  (deg) 

0 

+1-5 

+1-7 

Deviation  from  Land  A/S  @  Touchdown  (KEAS) 

0 

+1-5 

+/-  10 

Maximum  Touchdown  Sink  Rate  (ft/  sec) 

<4 

4 

6 

Deviation  from  RWY  heading  @  Touchdown  (deg) 

0 

+1-2 

+1-4 

FEEL  SYSTEM 

Baseline 
Aft  Stick  Gain  = 


TEST  ENGINEER/PILOT  NOT  FLYING 

Record  pilot  comments  and  CHR  for  each  run. 

Record  task  and  performance  parameters  at  simulated  touchdown. 


PILOT  TECHNIQUE 

Baseline 

Spoilers  in  at _ 


_ UNCLASSIFIED _ 

Figure  C-4  -  LAMARS  Test  Card  1 
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TASK  ID 


TASK 


UNCLASSIFIED 


HAVE  STAV 


STAV-1  XW  Precision  Landing 


FLIGHT  PHASE 

Landing 


TASK  DESCRIPTION 

Precision  Landing  from  Normal  Approach 


FIXED  PARAMETERS 

Wind:  15  knots  crosswind 
Config:  Gear  down,  30 
deg  spoiler  bias,  mid-fuel 
weight  with  payload 

RWY  surface:  Concrete 

Initial  Position:  4  NM  out 
Initial  Speed:  200  knots 
Initial  Heading:  On  LOC 
ALT:  1000  ft  AGL 
Landing  Speed:  Vapp-10 

GW:  232683  lb  for  STAV 

CG  pos  FS:  1044.14  in 

BL:  -0.04  in 

WL:  107.27 

VARIED  PARAMETERS 


TEST  PROCEDURE 

PILOT 

1.  Perform  normal  approach  and  landing  attitude  corrections. 

2.  Continue  to  fly  final  approach  until  flare. 

3.  Touchdown  target  is  designated  on  runway. 


EVALUATION  SEGMENT 

Precision  Landing  from  Normal  approach 


Approach  Airspeed: 

175 

195 

Control  System: 

Alpha 

Gamma 

Pitch  Rate 

Start  Evaluation:  600  ft  AGL,  Approach  speed,  Descending 

End  Evaluation:  Simulated  Touchdown 

1  EVALUATION  BASIS 

Evaluate  handling  qualities  when  landing  in  a  high-gain  task.  There  should  be  no 
tendency  to  bobble  in  pitch  or  roll  for  PIOs. 

PERFORMANCE  STANDARDS 

TARGET 

DESIRED 

ADEQUATE 

Landing  Zone  (for  ref.  point  btwn  main  gear)  (ft) 

Aim  Point 

50  x  500 

100 x 1000 

Maximum  Bank  Angle  Below  50  ft  AGL  (deg) 

0 

+1-5 

+1-7 

Deviation  from  Land  A/S  @  T ouchdown  (KEAS) 

0 

+1-5 

+/-  10 

Maximum  Touchdown  Sink  Rate  (ft /  sec) 

<4 

4 

6 

Deviation  from  RWY  heading  @  Touchdown  (deg) 

0 

+1-2 

+1-4 

FEEL  SYSTEM 

Baseline 

Aft  Stick  Gain  =  _ 


TEST  ENGINEER/PILOT  NOT  FLYING 

Record  pilot  comments  and  CHR  for  each  run. 

Record  task  and  performance  parameters  at  simulated  touchdown. 


PILOT  TECHNIQUE 

Baseline 

Spoilers  in  at _ 


Figure  C-5  -  LAMARS  Test  Card  2 
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TASK  ID 
STAV-2  CALM 


FLIGHT  PHASE 

Landing 


TASK 

Lateral  Offset  Landing 


UNCLASSIFIED 


TASK  DESCRIPTION 

PILOT 

Precision  Landing  from  Lateral  Offset 

FIXED  PARAMETERS 

Wind:  Calm 

Config:  Gear  down,  30 
deg  spoiler  bias,  mid-fuel 
weight  with  payload 

RWY  surface:  Concrete 

Initial  Position:  4  NM  out 
Initial  Speed:  200  knots 
Initial  Heading:  On  LOC 
ALT:  1000  ft  AGL 
Landing  Speed:  Vapp-10 

Lat  Offset:  +/-  200  ft  offset 
from  LOC 

Cloud  Breakout:  300  ft  AGL 

GW:  232683  lb  for  STAV 

CG  pos:  Same  as  1 

VARIED  PARAMETERS 


TEST  PROCEDURE 

PILOT 

1.  Perform  normal  approach  and  landing  attitude  corrections. 

2.  At  “cloud  breakout”,  visually  correct  course  to  recapture  runway  centerline. 

3.  Continue  to  fly  final  approach  until  flare. 

4.  Touchdown  target  is  designated  on  runway. 


HAVE  STAV 


EVALUATION  SEGMENT  - 

Precision  Landing  from  Lateral  Offset 

Start  Evaluation:  600  ft  AGL,  Approach  speed,  Descending 
End  Evaluation:  Simulated  Touchdown 

EVALUATION  BASIS 

Evaluate  handling  qualities  when  landing  in  a  high-gain  task.  There  should  be  no 
tendency  to  bobble  in  pitch  or  roll  for  PIOs. 


PERFORMANCE  STANDARDS 

TARGET 

DESIRED 

ADEQUATE 

Landing  Zone  (for  ref.  point  btwn  main  gear)  (ft) 

Aim  Point 

50  x  500 

100 x 1000 

Maximum  Bank  Angle  Below  50  ft  AGL  (deg) 

0 

+/  -  5 

+1-7 

Deviation  from  Land  A/S  @  Touchdown  (KEAS) 

0 

+1-5 

+/-  10 

Maximum  Touchdown  Sink  Rate  (ft/  sec) 

<4 

4 

6 

Deviation  from  RWY  heading  @  Touchdown  (deg) 

0 

+1-2 

+1-4 

FEEL  SYSTEM 

Baseline 

Aft  Stick  Gain  =  _ 


TEST  ENGINEER/PILOT  NOT  FLYING 

Record  pilot  comments  and  CHR  for  each  run. 

Record  task  and  performance  parameters  at  simulated  touchdown. 


PILOT  TECHNIQUE 

Baseline 

Spoilers  in  at _ 


Figure  C-6  -  LAMARS  Test  Card  3 
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TASK  ID 


TASK 


UNCLASSIFIED 


HAVE  STAV 


STAV-2  XW  Lateral  Offset  Landing 


FLIGHT  PHASE 

Landing 


TASK  DESCRIPTION 

Precision  Landing  from  Lateral  Offset 


FIXED  PARAMETERS 

Wind:  15  knots  crosswind 
Config:  Gear  down,  30 
deg  spoiler  bias,  mid-fuel 
weight  with  payload 

RWY  surface:  Concrete 

Initial  Position:  4  NM  out 
Initial  Speed:  200  knots 
Initial  Heading:  On  LOC 
ALT:  1000  ft  AGL 
Landing  Speed:  Vapp-10 

Lat  Offset:  +/-  200  ft  offset 
from  LOC 

Cloud  Breakout:  300  ft  AGL 
GW:  232683  lb  for  STAV 

CG  pos:  Same  as  1 

TEST  PROCEDURE 


PILOT 

1.  Perform  normal  approach  and  landing  attitude  corrections. 

2.  At  “cloud  breakout”,  visually  correct  course  to  recapture  runway  centerline. 

3.  Continue  to  fly  final  approach  until  flare. 

4.  Touchdown  target  is  designated  on  runway. 


EVALUATION  SEGMENT  - £1 

Precision  Landing  from  Lateral  Offset 


Start  Evaluation:  600  ft  AGL,  Approach  speed,  Descending 
End  Evaluation:  Simulated  Touchdown 

EVALUATION  BASIS 

Evaluate  handling  qualities  when  landing  in  a  high-gain  task.  There  should  be  no 


PERFORMANCE  STANDARDS 

TARGET 

DESIRED 

ADEQUATE 

Landing  Zone  (for  ref.  point  btwn  main  gear)  (ft) 

Aim  Point 

50  x  500 

100 x 1000 

Maximum  Bank  Angle  Below  50  ft  AGL  (deg) 

0 

+/  -  5 

+1-7 

Deviation  from  Land  A/S  @  Touchdown  (KEAS) 

0 

+/  -  5 

+/- 10 

Maximum  Touchdown  Sink  Rate  (ft/  sec) 

<4 

4 

6 

Deviation  from  RWY  heading  @  Touchdown  (deg) 

0 

+1-2 

+1-4 

FEEL  SYSTEM 

Baseline 


Aft  Stick  Gain  = 


TEST  ENGINEER/PILOT  NOT  FLYING 

PILOT  TECHNIQUE 

Record  pilot  comments  and  CHR  for  each  run. 

Baseline 

Record  task  and  performance  parameters  at  simulated  touchdown. 

Spoilers  in  at  AGL 

_ UNCLASSIFIED _ 

Figure  C-7  -  LAMARS  Test  Card  4 
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TASK  ID 


TASK 


UNCLASSIFIED 


HAVE  STAV 


STAV-3  CALM 

Vertical  Offset  Landing 

FLIGHT  PHASE 

TASK  DESCRIPTION 

PILOT 

DATE 

Landing 

Precision  Landing  from  Vertical  Offset 

FIXED  PARAMETERS 

Wind:  Calm 

Config:  Gear  down,  30 
deg  spoiler  bias,  mid-fuel 
weight  with  payload 

RWY  surface:  Concrete 

Initial  Position:  4  NM  out 
Initial  Speed:  200  knots 

Initial  Heading:  On  LOC 
ALT:  1000  ft  AGL 
Landing  Speed:  Vapp-10 

Long  Offset:  +  50  ft  vertical 
Offset  from  G/S  (0.5  dot  above) 
Cloud  Breakout:  300  ft  AGL 
GW:  232683  lb  for  STAV 

CG  pos:  Same  as  1 

TEST  PROCEDURE 

PILOT 

1 .  Perform  normal  approach  and  landing  attitude  corrections. 

2.  At  “cloud  breakout”,  visually  correct  glideslope  to  make  touchdown  at  target. 
Maintain  desired  vertical  velocity  and  approach  speed. 

3.  Continue  to  fly  final  approach  until  flare. 

4.  Touchdown  target  is  designated  on  runway. 


EVALUATION  SEGMENT  - 

Precision  Landing  from  Vertical  Offset 

Start  Evaluation:  600  ft  AGL,  Approach  speed,  Descending 
End  Evaluation:  Simulated  Touchdown 

EVALUATION  BASIS 

Evaluate  handling  qualities  when  landing  in  a  high-gain  task.  There  should  be  no 


PERFORMANCE  STANDARDS 

TARGET 

DESIRED 

ADEQUATE 

Landing  Zone  (for  ref.  point  btwn  main  gear)  (ft) 

Aim  Point 

50  x  500 

100 x 1000 

Maximum  Bank  Angle  Below  50  ft  AGL  (deg) 

0 

+1-5 

+1-7 

Deviation  from  Land  A/S  @  Touchdown  (KEAS) 

0 

+1-5 

+/-  10 

Maximum  Touchdown  Sink  Rate  (ft/  sec) 

<4 

4 

6 

Deviation  from  RWY  heading  @  Touchdown  (deg) 

0 

+1-2 

+1-4 

FEEL  SYSTEM 

Baseline 

Aft  Stick  Gain  =  _ 


TEST  ENGINEER/PILOT  NOT  FLYING 


PILOT  TECHNIQUE 


Record  pilot  comments  and  CHR  for  each  run.  Baseline 

Record  task  and  performance  parameters  at  simulated  touchdown.  Spoilers  in  at _ AGL 


_ UNCLASSIFIED _ 

Figure  C-8  -  LAMARS  Test  Card  5 
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TASK  ID 


TASK 


UNCLASSIFIED 


HAVE  STAV 


STAV-3  XW 

Vertical  Offset  Landing 

FLIGHT  PHASE 

TASK  DESCRIPTION 

PILOT 

DATE 

Landing 

Precision  Landing  from  Vertical  Offset 

FIXED  PARAMETERS 

Wind:  15  knots  crosswind 
Config:  Gear  down,  30 
deg  spoiler  bias,  mid-fuel 
weight  with  payload 

RWY  surface:  Concrete 

Initial  Position:  4  NM  out 
Initial  Speed:  200  knots 

Initial  Heading:  On  LOC 
ALT:  1000  ft  AGL 
Landing  Speed:  Vapp-10 

Long  Offset:  +  50  ft  vertical 
Offset  from  G/S  (0.5  dot  above) 
Cloud  Breakout:  300  ft  AGL 
GW:  232683  lb  for  STAV 

CG  pos:  Same  as  1 

TEST  PROCEDURE 

PILOT 

1 .  Perform  normal  approach  and  landing  attitude  corrections. 

2.  At  “cloud  breakout”,  visually  correct  glideslope  to  make  touchdown  at  target. 
Maintain  desired  vertical  velocity  and  approach  speed. 

3.  Continue  to  fly  final  approach  until  flare. 

4.  Touchdown  target  is  designated  on  runway. 


EVALUATION  SEGMENT  - 

Precision  Landing  from  Vertical  Offset 

Start  Evaluation:  600  ft  AGL,  Approach  speed,  Descending 
End  Evaluation:  Simulated  Touchdown 

EVALUATION  BASIS 

Evaluate  handling  qualities  when  landing  in  a  high-gain  task.  There  should  be  no 


PERFORMANCE  STANDARDS 

TARGET 

DESIRED 

ADEQUATE 

Landing  Zone  (for  ref.  point  btwn  main  gear)  (ft) 

Aim  Point 

50  x  500 

100 x 1000 

Maximum  Bank  Angle  Below  50  ft  AGL  (deg) 

0 

+1-5 

+1-7 

Deviation  from  Land  A/S  @  Touchdown  (KEAS) 

0 

+1-5 

+/-  10 

Maximum  Touchdown  Sink  Rate  (ft/  sec) 

<4 

4 

6 

Deviation  from  RWY  heading  @  Touchdown  (deg) 

0 

+1-2 

+1-4 

FEEL  SYSTEM 

Baseline 

Aft  Stick  Gain  =  _ 


TEST  ENGINEER/PILOT  NOT  FLYING 


PILOT  TECHNIQUE 


Record  pilot  comments  and  CHR  for  each  run.  Baseline 

Record  task  and  performance  parameters  at  simulated  touchdown.  Spoilers  in  at _ AGL 


_ UNCLASSIFIED _ 

Figure  C-9  -  LAMARS  Test  Card  6 
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Flight  Test  Cards 

The  following  test  cards  were  used  during  flight  testing  on  the  Total  In-Flight 
Simulator  (TIFS)  aircraft. 


UNCLASSIFIED 


HAVE  STAV 


TIFS-1  CALM 

Precision  Landing 

FLIGHT  PHASE 

TASK  DESCRIPTION 

PILOT 

DATE 

Landing 

Precision  Landing  from  Normal  2.5  deg  Approach 

FIXED  PARAMETERS 

Wind:  Calm 
Config:  Gear  down,  30 
deg  spoiler  bias,  mid-fuel 
weight  with  payload 
RWY  surface:  Concrete 


Initial  Position:  5  NM  out 
Initial  Speed:  185  knots 
Initial  Heading:  On  LOC 
ALT:  1000  ft  AGL 


':2326831b  for  STAV 
5  pos  FS:  1044.14  in 
BL:  -0.04  in 
WL:  107.27 


EVALUATION  SEGMENT 

Precision  Landing  from  Normal  approach 


VARIED  PARAMETERS 

Pilot: 

1 

2 

3 

Control  /  Feel  System: 

Baseline 

Secondary 

Start  Evaluation:  600  ft  AGL,  Approach  speed,  Descending 

End  Evaluation:  Simulated  Touchdown 

1  EVALUATION  BASIS 

Evaluate  handling  qualities  when  landing  in  a  high-gain  task.  There  should  be  no 
tendency  to  bobble  in  pitch  or  roll  for  PIOs. 

PERFORMANCE  STANDARDS 

TARGET 

DESIRED 

ADEQUATE 

Landing  Zone  (for  ref.  point  btwn  main  gear)  (ft) 

Lateral  Aim 

+/-  25 

+/-50 

Long  Aim 

+1000/-500 

+1500/-750 

Maximum  Bank  Angle  Below  50  ft  AGL  (deg) 

0 

+/  -  5 

+1-7 

Maximum  Touchdown  Sink  Rate  (ft/  sec) 

<4 

4 

6 

Deviation  from  RWY  heading  @  Touchdown  (deg) 

0 

+1-2 

+1-4 

FEEL  SYSTEM 

Baseline 

Long  Force  Gradient  =  5  x  Baseline  -  start  at  100  ft  above  touchdown  level 

PILOT  TECHNIQUE 


Record  pilot  comments  and  CHR  for  each  run.  Baseline 

Record  task  and  performance  parameters  at  simulated  touchdown.  Spoilers  in  at  30  ft  above  touchdown  level 


_ UNCLASSIFIED 

Figure  C-10  -  TIFS  Test  Card  1 
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TASK  ID 


TASK 


UNCLASSIFIED 


HAVE  STAV 


TIFS-1  XW 

Precision  Landing 

FLIGHT  PHASE 

TASK  DESCRIPTION 

PILOT 

DATE 

Landing 

Precision  Landing  from  Normal  2.5  deg  Approach 

FIXED  PARAMETERS 

Wind:  15  knots  crosswind 
Config:  Gear  down,  30 
deg  spoiler  bias,  mid-fuel 
weight  with  payload 

RWY  surface:  Concrete 

Initial  Position:  5  NM  out 
Initial  Speed:  185  knots 
Initial  Heading:  On  LOC 
ALT:  1000  ft  AGL 

GW:  232683  lb  for  STAV 

CG  pos  FS:  1044.14  in 

BL:  -0.04  in 

WL:  107.27 

EVALUATION  SEGMENT  - kJ 

Precision  Landing  from  Normal  approach 

Start  Evaluation:  600  ft  AGL,  Approach  speed,  Descending 
End  Evaluation:  Simulated  Touchdown 

EVALUATION  BASIS 

Evaluate  handling  qualities  when  landing  in  a  high-gain  task.  There  should  be  no 


PERFORMANCE  STANDARDS 

TARGET  DESIRED 

ADEQUATE 

Landing  Zone  (for  ref.  point  btwn  main  gear)  (ft) 

Lateral  Aim  +/-  25 

+/-50 

Long  Aim  +1000/-500 

+1500/-750 

Maximum  Bank  Angle  Below  50  ft  AGL  (deg) 

0  +/  -  5 

+1-7 

Maximum  Touchdown  Sink  Rate  (ft/  sec) 

<4  4 

6 

Deviation  from  RWY  heading  @  Touchdown  (deg) 

0  +1-2 

+1-4 

FEEL  SYSTEM 

Baseline 

Long  Force  Gradient  =  5  x  Baseline  -  start  at  100  ft  above  touchdown  level 

PILOT  TECHNIQUE 


Record  pilot  comments  and  CHR  for  each  run.  Baseline 

Record  task  and  performance  parameters  at  simulated  touchdown.  Spoilers  in  at  30  ft  above  touchdown  level 


_ UNCLASSIFIED 

Figure  C-ll  -  TIFS  Test  Card  2 
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TASK  ID 


TASK 


UNCLASSIFIED 


HAVE  STAV 


TIFS-2  CALM 

Lateral  Offset  Landing 

FLIGHT  PHASE 

TASK  DESCRIPTION 

PILOT 

DATE 

Landing 

Precision  Landing  from  Lateral  Offset  2.5  deg  Approach 

FIXED  PARAMETERS 

Wind:  Calm 

Config:  Gear  down,  30 
deg  spoiler  bias,  mid-fuel 
weight  with  payload 

RWY  surface:  Concrete 

Initial  Position:  5  NM  out 
Initial  Speed:  185  knots 
Initial  Heading:  On  LOC 
ALT:  1000  ft  AGL 

Lat  Offset:  +/-  200  ft  offset 
from  LOC 

Cloud  Breakout:  300  ft  AGL 
GW:  232683  lb  for  STAV 

CG  pos:  Same  as  1 

EVALUATION  SEGMENT  - 

Precision  Landing  from  Lateral  Offset 

Start  Evaluation:  600  ft  AGL,  Approach  speed,  Descending 
End  Evaluation:  Simulated  Touchdown 

EVALUATION  BASIS 

Evaluate  handling  qualities  when  landing  in  a  high-gain  task.  There  should  be  no 


PERFORMANCE  STANDARDS 

TARGET  DESIRED 

ADEQUATE 

Landing  Zone  (for  ref.  point  btwn  main  gear)  (ft) 

Lateral  Aim  +/-  25 

+/-50 

Long  Aim  +1000/-500 

+1500/-750 

Maximum  Bank  Angle  Below  50  ft  AGL  (deg) 

0  +/  -  5 

+1-7 

Maximum  Touchdown  Sink  Rate  (ft/  sec) 

<4  4 

6 

Deviation  from  RWY  heading  @  Touchdown  (deg) 

0  +1-2 

+1-4 

FEEL  SYSTEM 

Baseline 

Long  Force  Gradient  =  5  x  Baseline  -  start  at  100  ft  above  touchdown  level 

PILOT  TECHNIQUE 


Record  pilot  comments  and  CHR  for  each  run.  Baseline 

Record  task  and  performance  parameters  at  simulated  touchdown.  Spoilers  in  at  30  ft  above  touchdown  level 


_ UNCLASSIFIED 

Figure  C-12  -  TIFS  Test  Card  3 
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TASK  ID 


TASK 


UNCLASSIFIED 


HAVE  STAV 


TIFS-2  XW 

Lateral  Offset  Landing 

FLIGHT  PHASE 

TASK  DESCRIPTION 

PILOT 

DATE 

Landing 

Precision  Landing  from  Lateral  Offset  2.5  deg  Approach 

FIXED  PARAMETERS 

Wind:  15  knots  crosswind 
Config:  Gear  down,  30 
deg  spoiler  bias,  mid-fuel 
weight  with  payload 

RWY  surface:  Concrete 

Initial  Position:  5  NM  out 
Initial  Speed:  185  knots 
Initial  Heading:  On  LOC 
ALT:  1000  ft  AGL 

Lat  Offset:  +/-  200  ft  offset 
from  LOC 

Cloud  Breakout:  300  ft  AGL 
GW:  232683  lb  for  STAV 

CG  pos:  Same  as  1 

EVALUATION  SEGMENT  - 

Precision  Landing  from  Lateral  Offset 

Start  Evaluation:  600  ft  AGL,  Approach  speed,  Descending 
End  Evaluation:  Simulated  Touchdown 

EVALUATION  BASIS 

Evaluate  handling  qualities  when  landing  in  a  high-gain  task.  There  should  be  no 


PERFORMANCE  STANDARDS 

TARGET  DESIRED 

ADEQUATE 

Landing  Zone  (for  ref.  point  btwn  main  gear)  (ft) 

Lateral  Aim  +/-  25 

+/-50 

Long  Aim  +1000/-500 

+1500/-750 

Maximum  Bank  Angle  Below  50  ft  AGL  (deg) 

0  +/  -  5 

+1-7 

Maximum  Touchdown  Sink  Rate  (ft/  sec) 

<4  4 

6 

Deviation  from  RWY  heading  @  Touchdown  (deg) 

0  +1-2 

+1-4 

FEEL  SYSTEM 

Baseline 

Long  Force  Gradient  =  5  x  Baseline  -  start  at  100  ft  above  touchdown  level 

PILOT  TECHNIQUE 


Record  pilot  comments  and  CHR  for  each  run.  Baseline 

Record  task  and  performance  parameters  at  simulated  touchdown.  Spoilers  in  at  30  ft  above  touchdown  level 


_ UNCLASSIFIED 

Figure  C-13  -  TIFS  Test  Card  4 
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Appendix  D  -  TIFS  Flight  Test  Matrix 


Following  the  simulator  testing  in  the  Large  Amplitude  Multimode  Aerospace 
Simulator  (LAMARS),  the  conditions  that  warranted  further  evaluation  were  selected  for 
flight  testing  in  the  Total  In-Flight  Simulator  (TIFS).  The  matrix  below  shows  the  actual 
flight  test  runs. 

Table  D-l  -  TIFS  Flight  Test  Matrix 


Key  of  Abbreviations  in  Modeling  and  Simulation  Matrix 

Pilot 

1  Speares 

2  Domsalla 

3  Quashnock 

Task 

N  Normal 

L  Lateral  Offset 

(P)  Practice 

Feel  System 

B  Baseline 

Feel  System 

0  LAMARS  Optimized 

Hour  # 

Pilot 

Required  to 
Meet  Objective 

Control 

Type 

Feel 

System 

Approach 

Airspeed 

Task 

Crosswind 

1-1 

1 

1,2  and  3 

Alpha 

B 

185  KIAS 

N(P) 

0 

1-2 

1 

1,2  and  3 

Alpha 

B 

185  KIAS 

N 

0 

1-3 

1 

1,2  and  3 

Alpha 

B 

185  KIAS 

N 

7 

1-4 

1 

1,2  and  3 

Alpha 

B 

185  KIAS 

L(P) 

0 

1-5 

1 

1,2  and  3 

Alpha 

B 

185  KIAS 

L 

0 

2-1 

1 

1,2  and  3 

Alpha 

B 

185  KIAS 

L 

7 

2-2 

1 

2  and  3 

Alpha 

O 

185  KIAS 

N(P) 

0 

2-3 

1 

2  and  3 

Alpha 

O 

185  KIAS 

N 

0 

2-4 

1 

2  and  3 

Alpha 

O 

185  KIAS 

N 

7 

2-5 

1 

2  and  3 

Alpha 

O 

185  KIAS 

L(P) 

0 

2-6 

1 

2  and  3 

Alpha 

O 

185  KIAS 

L 

0 

2-7 

1 

2  and  3 

Alpha 

O 

185  KIAS 

L 

7 

2-8 

1 

2  and  3 

Alpha 

O 

185  KIAS 

N 

7 

3-1 

2 

1,2  and  3 

B 

185  KIAS 

N(P) 

0 

3-2 

2 

1,2  and  3 

Alpha 

B 

185  KIAS 

N 

0 

3-3 

2 

1,2  and  3 

Alpha 

B 

185  KIAS 

N 

7 

3-4 

2 

1,2  and  3 

Alpha 

B 

185  KIAS 

UP) 

0 

3-5 

2 

1,2  and  3 

B 

185  KIAS 

L 

0 

4-1 

2 

2  and  3 

mm mu 

O 

185  KIAS 

N(P) 

0 

4-2 

2 

2  and  3 

m m 

O 

185  KIAS 

N 

0 

4-3 

2 

2  and  3 

Alpha 

O 

185  KIAS 

N 

7 

4-4 

2 

2  and  3 

Alpha 

O 

185  KIAS 

UP) 

0 

4-5 

2 

2  and  3 

Alpha 

O 

185  KIAS 

L 

0 

4-6 

2 

2  and  3 

Alpha 

O 

185  KIAS 

L 

7 

4-7 

2 

1,2  and  3 

Alpha 

B 

185  KIAS 

L 

7 

5-1 

3 

1,2  and  3 

Alpha 

B 

185  KIAS 

N(P) 

0 

5-2 

3 

1,2  and  3 

Alpha 

B 

185  KIAS 

N 

0 

5-3 

3 

1,2  and  3 

Alpha 

B 

185  KIAS 

N 

7 

5-4 

3 

1,2  and  3 

Alpha 

B 

185  KIAS 

UP) 

0 

5-5 

3 

1,2  and  3 

Alpha 

B 

185  KIAS 

L 

0 

5-6 

3 

1,2  and  3 

B 

185  KIAS 

L 

7 
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Table  D-l  -  TIFS  Flight  Test  Matrix  (Continued) 


Hour  # 

Pilot 

Required  to 
Meet  Objective 

Control 

Type 

Feel 

System 

Approach 

Airspeed 

Task 

Crosswind 

6-1 

3 

2  and  3 

Alpha 

O 

185  KIAS 

N(P) 

0 

6-2 

3 

2  and  3 

Alpha 

O 

185  KIAS 

N 

0 

6-3 

3 

2  and  3 

Alpha 

o 

185  KIAS 

N 

7 

6-4 

3 

2  and  3 

Alpha 

o 

185  KIAS 

L(P) 

0 

6-5 

3 

2  and  3 

Alpha 

o 

185  KIAS 

L 

0 

6-6 

3 

2  and  3 

Alpha 

o 

185  KIAS 

L 

7 

6-7 

3 

N/A 

Alpha 

B 

185  KIAS 

N 

0 

7-1 

3 

N/A 

Alpha 

B 

185  KIAS 

N 

0 

7-2 

1 

N/A 

Alpha 

B 

185  KIAS 

N 

0 

7-3 

1 

N/A 

Alpha 

B 

185  KIAS 

N 

0 

7-4 

1 

N/A 

Alpha 

O 

185  KIAS 

N 

0 

7-5 

1 

N/A 

Alpha 

O 

185  KIAS 

N 

0 

7-6 

1 

N/A 

Alpha 

B 

185  KIAS 

N 

0 

7-7 

1 

N/A 

Alpha 

B 

185  KIAS 

N 

0 

7-8 

1 

N/A 

Alpha 

O 

185  KIAS 

N 

0 

8-1 

2 

N/A 

B 

185  KIAS 

N 

0 

8-2 

2 

N/A 

Alpha 

B 

185  KIAS 

N 

0 

8-3 

2 

N/A 

Alpha 

O 

185  KIAS 

N 

0 

8-4 

2 

N/A 

Alpha 

O 

185  KIAS 

N 

0 

8-5 

2 

N/A 

Alpha 

B 

185  KIAS 

N 

0 

8-6 

2 

N/A 

Alpha 

B 

185  KIAS 

N 

0 

8-7 

2 

N/A 

Alpha 

O 

185  KIAS 

N 

0 

OO 

i 

OO 

2 

N/A 

Alpha 

O 

185  KIAS 

N 

0 

8-9 

2 

N/A 

Alpha 

O 

185  KIAS 

N 

0 

9-1 

3 

N/A 

Alpha 

O 

185  KIAS 

N 

0 

9-2 

3 

N/A 

Alpha 

O 

185  KIAS 

N 

0 

9-3 

3 

N/A 

Alpha 

B 

185  KIAS 

N 

0 

9-4 

3 

N/A 

Alpha 

B 

185  KIAS 

N 

0 

9-5 

3 

N/A 

Alpha 

O 

185  KIAS 

N 

0 

9-6 

3 

N/A 

Alpha 

O 

185  KIAS 

N 

0 

9-7 

3 

N/A 

Alpha 

O 

185  KIAS 

N 

0 

9-8 

3 

N/A 

Alpha 

B 

185  KIAS 

N 

0 

9-9 

3 

N/A 

O 

185  KIAS 

N 

0 

10-1 

1 

N/A 

B 

185  KIAS 

N 

0 

10-2 

1 

N/A 

Alpha 

B 

185  KIAS 

N 

0 

10-3 

1 

N/A 

O 

185  KIAS 

N 

0 

174 


Appendix  E  -  Additional  TIFS  Flight  Test  Results 


Comparison  of  Baseline  and  Optimized  Feel  Systems  During  Lateral  Offset 


Data  Basis:  Flight  Test 

Test  Aircraft:  NC-131H  (TIFS)  /  N793VS 
-  Configuration:  Baseline  and  Optimized 
STAV  Version  2 

Test  Dates:  10-13  September  2007 

□  Baseline  Feel  System 

0  Optimized  Feel  System 

1 

■ 

1 

123456789  10 


Cooper  Harper  Rating 


Figure  E-l  -  Comparison  of  Baseline  and  Optimized  Systems  during  Lateral  Offset 
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Comparison  of  Baseline  and  Optimized  Feel  System  During  Precision  Landing 


14 


12 


10 


Data  Basis:  Flight  Test 
Test  Aircraft:  NC-131H  (TIFS)  /  N793VS 
Configuration:  Baseline  and  Optimized 
STAV  Version  2 

Test  Dates:  10-13  September  2007 


□  Baseline  Feel  System 
■  Optimized  Feel  System 


5  6  7 

Cooper  Harper  Rating 


10 


Figure  E-2  -  Comparison  of  Baseline  and  Optimized  Systems  during  Precision  Landing 
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Comparison  of  Baseline  and  Optimized  Feel  Systems  for  Pilot  1 


<V 

■Q 

£ 
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Data  Basis:  Flight  Test 

Test  Aircraft:  NC-131H  (TIFS)/ N793VS 
-  Configuration:  Baseline  and  Optimized 

STAV  Version  2 

Test  Dates:  10. 12  and  13  September  2007 

□  Baseline  Feel  System 
■  Optimized  Feel  System 

123456789  10 


Cooper  Harper  Rating 


Figure  E-3  -  Comparison  of  Baseline  and  Optimized  Systems  for  Pilot  1 
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Comparison  of  Baseline  and  Optimized  Feel  Systems  for  Pilot  2 


10 


Data  Basis:  Flight  Test 
Test  Aircraft:  NC-1 31 H  (TIFS)  /  N793VS 
Configuration:  Baseline  and  Optimized 
STAV  Version  2 

Test  Dates:  11, 12  and  13  September  2007 


5  6 


0) 

-Q 

£ 

Z  4 


m 


□  Baseline  Feel  System 
U  Optimized  Feel  System 


5  6  7 

Cooper  Harper  Rating 


10 


Figure  E-4  -  Comparison  of  Baseline  and  Optimized  Systems  for  Pilot  2 
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Comparison  of  Baseline  and  Optimized  Feel  System  Results  for  Pilot  3 


d> 

Si 

E 

3 

z 


Data  Basis:  Flight  Test 

Test  Aircraft:  NC-131H  (TIFS)/  N793VS 
Configuration:  Baseline  and  Optimized 

STAV  Version  2 

Test  Dates:  11, 12  and  13  September  2007 

□  Baseline  Feel  System 
■  Optimized  Feel  System 

123456789  10 


Cooper  Harper  Rating 


Figure  E-5  -  Comparison  of  Baseline  and  Optimized  Systems  for  Pilot  3 
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PIO  Rating  Comparison  of  Baseline  and  Optimized  Systems 


40 


35 


30 


25 


20 


15 


10 


Data  Basis:  Flight  Test 
Test  Aircraft:  NC-131 H  (TIFS)  /  N793VS 
Configuration:  Baseline  and  Optimized 
STAV  Version  2 

T est  Dates:  10-13  September  2007 


□  Baseline  Feel  System 
■  Optimized  Feel  System 


Turbulence  induced 
undesirable  pitch 
motions  that  did  not 
affect  task 
performance 


Pitch  correction 
overshoot  at 

10'AGL  and  tight 

control  led  to  pitch 
oscillations  that  did 
not  diverge 

I  I 


I  I 


3  4 

PIO  Rating 


Figure  E-6  -  PIO  Rating  Comparison  of  Baseline  and  Optimized  Systems 
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Table  E-l  -  Inadequate  Landing  Details  for  Baseline  and  Optimized  Systems 


17  Baseline  Landings  with  Inadequate  Results 


Flight  # 

Record  # 

Reasons  for  Inadequate  Results 

Length 

ROD 

Pitch 

1 

6 

ROD:  8.1;  A/S:  158 

X 

1 

16 

ROD:  6.2 

X 

1 

19 

Length:  1636 

X 

2 

8 

Pitch:  15.7;  A/S:  147 

X 

2 

11 

Pitch:  15.8;  A/S:  138 

X 

2 

14 

ROD:  9.8;  Pitch  15.2;  A/S:  134 

X 

X 

2 

20 

Length:  1599;  ROD:  8.4;  A/S:  137 

X 

X 

3 

22 

Length:  1728 

X 

3 

23 

ROD:  8.6;  Pitch:  15.4;  A/S:  160 

X 

X 

3 

26 

Pitch:  15.0;  A/S:  162 

X 

3 

27 

Pitch:  18.1 

X 

4 

21 

ROD:  7.7 

X 

4 

39 

ROD:  9.5 

X 

4 

42 

ROD:  9.0 

X 

4 

43 

ROD:  6.4 

X 

5 

5 

ROD:6.3 

X 

6 

5 

Length:  2529 

X 

Total 

4 

10 

6 

12  Optimized  Landings  with  Inadequate  Results 

Flight  # 

Record  # 

Reasons  for  Inadequate  Results 

Length 

ROD 

Pitch 

1 

26 

Length:  2101 

X 

1 

28 

ROD:  8.9 

X 

1 

31 

ROD:  6.5 

X 

3 

5 

Length:  2197;  A/S:  163 

X 

3 

8 

Length:  2343 

X 

3 

11 

ROD:  10.9 

X 

3 

20 

Length:  1622 

X 

4 

21 

ROD:  7.7 

X 

4 

40 

Length:  2104 

X 

4 

44 

ROD:  6.8 

X 

5 

17 

Length:  2079 

X 

5 

25 

ROD:  7.1 

X 

Total 

6 

6 

0 
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Appendix  F  -  Lessons  Learned 


Throughout  the  course  of  this  thesis,  there  were  several  lessons  learned  that 
should  be  highlighted.  Most  of  these  lessons  learned  are  specific  to  an  Air  Force 
Institute  of  Technology-Test  Pilot  School  (AFIT-TPS)  thesis,  as  they  were  garnered 
during  the  execution  of  a  TPS  Test  Management  Project  (TMP).  However,  the  general 
ideas  governing  test  planning  and  execution  can  be  applied  to  any  research  that  involves 
testing,  and  it  is  left  to  the  reader  to  apply  these  ideas  to  future  projects. 

When  attempting  to  define  a  subject  to  conduct  thesis  research  on,  an  AFIT 
student  should  use  all  of  the  local  resources  available  to  find  a  topic  of  interest.  This  not 
only  includes  AFIT  instructors  and  advisors,  but  also  extends  to  other  facilities  on  base. 
AFIT  should  continue  to  improve  and  expand  its  relationship  with  the  Air  Force 
Research  Laboratory.  There  are  a  myriad  of  topics  available  for  research  at  this  facility, 
but  it  remains  a  relatively  untapped  resource  for  the  majority  of  the  AFIT  student 
population.  In  addition  to  the  current  thesis  symposium  where  instructors  brief  students 
on  their  topics  of  interest,  AFIT  should  invite  other  facilities  on  base  to  brief  the  students 
on  areas  of  potential  research.  Although  this  research  may  initially  require  an  AFIT 
advisor  to  participate  in  a  field  they  have  not  been  involved  in  before,  it  would  provide 
an  excellent  opportunity  to  broaden  one’s  overall  engineering  experience. 

Test  management  projects  that  can  potentially  be  accomplished  off-station  should 
be  run  through  a  costs  and  benefits  analysis  to  determine  if  the  decision  to  conduct  the 
TMP  while  at  an  off-base  facility  makes  sense,  from  both  a  technical  and  risk  standpoint. 
Conducting  the  TMP  flight  testing  away  from  Edwards  carries  significant  risk,  in  the  fact 
that  the  schedule  is  constrained  by  TPS  scheduling  requirements.  The  maximum 
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realistic  time  away  is  one  five-day  work  week.  When  possible,  the  weekends  should  be 
used  to  travel  to  minimize  the  impact  on  the  TPS  schedule  and  to  acclimatize  the  test 
team  to  the  new  conditions,  especially  if  there  is  a  significant  time  change  involved.  The 
flight  test  schedule  is  put  at  risk  by  both  weather  and  maintenance  factors,  which  could 
effectively  prevent  or  at  best  severely  limit  the  number  of  flight  test  sorties 
accomplished.  However,  the  benefits  of  having  contractor  facilities,  personnel,  and 
equipment  on  site  minimizes  some  of  the  maintenance  risk,  while  scheduling  the  testing 
according  to  predicted  weather  patterns  can  reduce  the  weather  risk.  Try  to  front-load 
the  schedule  as  much  as  possible  to  allow  for  any  potential  flight  test  delays.  This  may 
entail  early  morning  take-offs  and  triple  turns,  but  the  test  team  must  be  flexible.  If  the 
testing  is  going  to  involve  traffic  pattern  work,  then  testing  at  an  offsite  location  with 
minimal  traffic  can  increase  the  amount  of  data  collected  and  minimize  the  impact  of  air 
traffic  control.  The  test  team  can  also  focus  all  of  their  efforts  on  the  project,  and  not 
worry  about  other  TPS  syllabus  events. 

When  possible,  simulations  of  the  flight  testing  should  be  accomplished  prior  to 
the  actual  flight  testing.  This  forces  the  test  team  to  create  test  cards  and  run  them,  so 
that  any  mistakes  can  be  worked  out  prior  to  wasting  flight  test  time.  It  also  allows  the 
test  team  to  practice  the  cadence  of  the  testing  itself,  so  that  all  evaluator  pilot  and  test 
conductor  duties  are  clearly  understood  before  testing  begins.  Testing  in  a  simulator 
allows  the  test  team  to  create  data  analysis  and  reduction  tools,  something  that  can 
streamline  the  actual  flight  data  reduction.  This  is  particularly  valuable  when  testing  on  a 
tight  schedule,  because  a  quick-look  at  the  data  can  allow  small  modifications  to  be  made 
to  the  testing,  something  that  could  not  be  accomplished  if  all  data  reduction  was  saved 
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until  after  flight  test.  Finally,  it  is  imperative  that  the  test  team  integrate  with  the 
simulator  technicians  early  in  the  test  process.  A  team  of  technicians  that  is  intimately 
familiar  with  the  test  program  provides  better  adaptability  when  test  procedures  must  be 
altered  or  simulator  problems  arise.  The  Air  Force  Research  Laboratory  Large 
Amplitude  Multimode  Aerospace  Research  Simulator  (LAMARS)  technicians  provided 
exemplary  support  throughout  the  project,  and  provide  a  fantastic  example  of  properly 
conducted  simulator  testing. 

When  conducting  tests,  the  test  team  must  always  remember  who  retains  test 
control.  The  test  team  must  reference  the  test  plan,  especially  when  testing  is  not 
proceeding  as  planned  or  when  actual  results  do  not  match  predictions.  This  will  help  to 
prevent  the  test  objectives  from  evolving  during  testing. 

Contracting  issues  should  be  accomplished  as  early  in  the  TMP  process  as 
possible.  When  dealing  with  multiple  contractors,  it  can  be  very  easy  to  lose  the  scope 
of  the  testing  and  become  bogged  down  in  the  paperwork.  Contracts  should  be  provided 
to  and  reviewed  by  the  test  team,  to  ensure  that  no  important  factors  are  omitted  (i.e.  who 
pays  for  the  fuel). 

Whenever  possible,  try  to  have  the  contractors  attend  the  test  plan  working  group 
and  technical  review  board  in  person.  It  is  much  easier  to  discuss  technical  procedures 
face  to  face  than  it  is  via  a  teleconference.  The  risk  of  a  miscommunication  in  testing 
procedure  or  capability  is  much  higher  when  conducting  all  meetings  remotely. 

Finally,  the  test  team  must  take  model  limitations  into  account  during  testing,  and 
must  be  flexible  in  their  test  design  to  account  for  unforeseen  changes  in  the  model. 
Current  Supersonic  Tailless  Air  Vehicle  (STAY)  model  predictions  were  based  on  a 
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constant  center  of  gravity  location  and  aircraft  configuration,  and  testing  was  designed  to 
take  this  into  account.  The  instantaneous  center  of  rotation  was  initially  thought  to  be  in 
front  of  the  actual  aircraft,  and  the  test  team  expected  the  pilots  to  feel  a  motion  that  was 
opposite  the  initial  inceptor  input.  However,  the  pilots  did  not  perceive  this  motion 
during  simulator  testing.  After  this  simulator  testing  was  conducted,  it  was  discovered 
that  the  previous  location  for  instantaneous  center  of  rotation  was  incorrect.  The  correct 
instantaneous  center  of  rotation  was  nearly  collocated  with  the  cockpit,  and  explained  the 
motions  perceived  by  the  pilots.  The  design  of  the  test  plan  and  objectives  minimized 
the  impact  of  this  change,  and  allowed  the  team  to  proceed  with  flight  testing  without 
altering  the  test  plan. 
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Appendix  G  -  Pilot  Pool  Information 


Table  G-l  -  ICS  Pilot  Information 


Pilot 

Service 

Aircraft 

Flight 

Time 

Test 

Experience 

1 

USMC 

F/A-18C 

1200 

N 

2 

USN 

P-3C 

2500 

N 

3 

USAF 

B-1B 

1350 

N 

4 

USAF 

F-16 

1300 

N 

5 

USAF 

F-15/F-117 

2100 

Y 

6 

USAF 

B-l/B-2 

1400 

N 

7 

USN 

P-3C 

1500 

N 

8 

USAF 

F-16/F-117 

2200 

N 

9 

USAF 

F-15E 

1500 

N 

10 

USAF 

F-15C 

1300 

N 

11 

USAF 

C-130 

2900 

N 

12 

USAF 

F-15C 

1800 

Y 

13 

USAF 

F-15E 

2100 

Y 

14 

USAF 

F-15E 

850 

N 

15 

USN 

EA-6B 

900 

N 

16 

USN 

SH-60B 

950 

N 

17 

Civilian 

Civil 

2000 

N 

18 

USN 

P-3C 

1000 

N 

19 

USAF 

F-15E 

1065 

N 

Table  G-2  -  LAMARS  Pilot  Information 


Pilot 

Service 

Aircraft 

Flight 

Time 

Test 

Experience 

1 

USAF 

F-15E 

1200 

Y 

2 

USAF 

A-10 

1200 

Y 

3 

USAF 

C-9/  C-17 

2500 

Y 

4 

USAF 

F-15/  F-117 

3000 

Y 

Table  G-3  -  TIFS  Pilot  Information 


Pilot 

Service 

Aircraft 

Flight 

Time 

Test 

Experience 

1 

USAF 

F-15E 

1200 

Y 

2 

USAF 

A-10 

1200 

Y 

3 

USAF 

C-9 /  C-17 

2500 

Y 

4 

USAF 

F-15/  F-117 

3000 

Y 
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